Main Page: Difference between revisions

From imde.io

 
(181 intermediate revisions by 3 users not shown)
Line 1: Line 1:
==What is IMDE==
==What is IMDE?==
Interoperable Modular Data Exchange (IMDE) is a framework that will enable machine-2-machine exchange data across the entire value network. Minimizing the cost for collecting, using and distributing data. IMDE can overcome a number of the challenges faced by organizations, [[Solution_Providers|solution providers]] and data processors like [[DataPools_and_Data_Networks|datapools]] in terms of improving simplification of data exchange while at the same time improving data quality (consistency, relevance, completeness, accuracy and timeliness). The biggest benefits are in exchanging data with both upstream and downstream business partners. The beneficiaries are machines!, so not humans. The whole framework is designed for fully autonomous communication between machines.
Interoperable Modular Data Exchange (IMDE) is a framework that will enable machine-2-machine exchange data across the entire value network. Minimizing the cost for collecting, using and distributing data. IMDE can overcome a number of the challenges faced by organizations, [[Solution_Providers|solution providers]] and data processors like [[DataPools_and_Data_Networks|datapools]] in terms of improving simplification of data exchange while at the same time improving data quality (consistency, relevance, completeness, accuracy and timeliness). The biggest benefits are in exchanging data with both upstream and downstream business partners. The beneficiaries are machines!, so not humans. The whole framework is designed for fully autonomous communication between machines.


==Why Interoperable and Modular?==
==Why Interoperable and Modular?==
Many of the initiatives in the past focussed on creating standards for specific use cases, or in specific industries. These standards often covered all three layers of data exchange:
Many of the initiatives in the past focussed on creating standards for specific use cases, or in specific industries. These standards often covered all six layers of data exchange:
* '''Information Model''', this covers taxonomy and data definitions, focussing on what do we mean with what data point. Examples of information models are: ETIM, GTS-Global Textile Scheme and GS1-Global Data Model)
* '''Data Format'''. This layer covers the technical way the data message or file is constructed and the related file/message type. For example, an Excel template for file-based data exchange or a XML-XSD for message-based data exchange.
* '''Data Exchange''': The technical way the data is exchanged between usually two systems, but can also be between systems and humans. Examples are APIs (Rest / Soap), DataPorts, file transfer (FTP, sFTP, FTPs, AS2, Dropbox, Box) and email. Data Exchange can be peer-to-peer (DataPorts) or via Data Pools/Networks (2BA, PS in Food service, GTS, GS1-GDSN)


The challenge in this approach is that the world is big, so there a many standards, for different territories, industries and use cases. Which has resulted in the fact that data exchange is still a technical and organizational nightmare. Example: A Food manufacturer sourcing raw materials and components from multiple suppliers and selling both A-Brand and Private Label products can be dealing with dozens of  different ways to exchange origin and allergen data. While the type of data is a 100% the same in all situations: Allergens information has the same data points for raw materials, semifinished goods and finished goods, and allergens data is not different for A-Brands or Private label brands. But still, from a technical standpoint, organizations need to deal with many ways to export or import this kind of data.
{| class="wikitable" style="width: 100%;"
|+ IMDE - 6 Layer Framework for Interoperable Modular Data Exchange


The solution is to move away from all-in-one standards, where all standards had the strategy to become THE standard. And where every standard covers all 3 layers: the exchange technology, the message structure and the information model. This new approach is based on the idea of allowing modular standards with independent solutions for all 3 layers.
|-
! style="color:white; background-color:#273247; border:solid 1px white; padding: 10px; vertical-align:top;" | L1
| style="color:#273247; background-color:#39b8bb; border:solid 1px white; padding: 10px; font-weight:bold; vertical-align:top;" | TX
| style="color:#273247; background-color:#97dbdd; border:solid 1px white; padding: 10px; width: 200px; text-align:left; font-weight:bold; vertical-align:top;" | Taxonomy
| style="color:#273247; background-color:#f3f3f3; border:solid 1px white; padding: 10px; font-weight:normal; text-align:left; vertical-align:top;" | Establishes a unified communication language to ensure consistent terminology and understanding across different systems and platforms.
| style="color:white; background-color:white; border:solid 1px white; padding: 10px; vertical-align:top;" | [[ :Category:Taxonomy| Link]]
|-
! style="color:white; background-color:#273247; border:solid 1px white; padding: 10px; vertical-align:top;" | L2
| style="color:#273247; background-color:#39b8bb; border:solid 1px white; padding: 10px; font-weight:bold; vertical-align:top;" | IM
| style="color:#273247; background-color:#97dbdd; border:solid 1px white; padding: 10px; width: 200px; text-align:left; font-weight:bold; vertical-align:top;" | Information Model
| style="color:#273247; background-color:#f3f3f3; border:solid 1px white; padding: 10px; font-weight:normal; text-align:left; vertical-align:top;" |Defines the structure and relationships of data elements, providing a common blueprint for organizing and interpreting data.
| style="color:white; background-color:white; border:solid 1px white; padding: 10px; vertical-align:top;"| [[ :Category:Information_Model| Link]]
|-
! style="color:white; background-color:#273247; border:solid 1px white; padding: 10px; vertical-align:top;" |L3
| style="color:#273247; background-color:#39b8bb; border:solid 1px white; padding: 10px; font-weight:bold; vertical-align:top;" | ID
| style="color:#273247; background-color:#97dbdd; border:solid 1px white; padding: 10px; width: 200px; text-align:left; font-weight:bold; vertical-align:top;" | Identifiers
| style="color:#273247; background-color:#f3f3f3; border:solid 1px white; padding: 10px; font-weight:normal; text-align:left; vertical-align:top;" | Supports various standards for uniquely identifying entities such as locations, products, brands and organizations.  
| style="color:white; background-color:white; border:solid 1px white; padding: 10px; vertical-align:top;" | [[ID_-_Identifier_Types|Link]]
|-
! style="color:white; background-color:#273247; border:solid 1px white; padding: 10px; vertical-align:top;" | L4
| style="color:#273247; background-color:#39b8bb; border:solid 1px white; padding: 10px; font-weight:bold; vertical-align:top;" | CL
| style="color:#273247; background-color:#97dbdd; border:solid 1px white; padding: 10px; width: 200px; text-align:left; font-weight:bold; vertical-align:top;" | Code Lists
| style="color:#273247; background-color:#f3f3f3; border:solid 1px white; padding: 10px; font-weight:normal; text-align:left; vertical-align:top;" | Standardizes sets of codes used to represent specific data elements, ensuring consistency and interoperability across different datasets and platforms. Example: Unit of Measure
| style="color:white; background-color:white; border:solid 1px white; padding: 10px; vertical-align:top;" | [[Layer_-_Code_Lists | Link]]
|-
! style="color:white; background-color:#273247; border:solid 1px white; padding: 10px; vertical-align:top;" |L5
| style="color:#273247; background-color:#39b8bb; border:solid 1px white; padding: 10px; font-weight:bold; vertical-align:top;" | DF
| style="color:#273247; background-color:#97dbdd; border:solid 1px white; padding: 10px; width: 200px; text-align:left; font-weight:bold; vertical-align:top;" | Data Formats
| style="color:#273247; background-color:#f3f3f3; border:solid 1px white; padding: 10px; font-weight:normal; text-align:left; vertical-align:top;" | Accommodates multiple data formatting standards in formats like JSON, XML, or spreadsheet templates, with clear declarations for each data container to facilitate seamless data exchange.
| style="color:white; background-color:white; border:solid 1px white; padding: 10px; vertical-align:top;" | [[Layer_-_Data_Format|Link]]
|-
! style="color:white; background-color:#273247; border:solid 1px white; padding: 10px; vertical-align:top;" | L6
| style="color:#273247; background-color:#39b8bb; border:solid 1px white; padding: 10px; font-weight:bold; vertical-align:top;" | DX
| style="color:#273247; background-color:#97dbdd; border:solid 1px white; padding: 10px; width: 200px; text-align:left; font-weight:bold; vertical-align:top;" | Data Exchange
| style="color:#273247; background-color:#f3f3f3; border:solid 1px white; padding: 10px; font-weight:normal; text-align:left; vertical-align:top;" | Defines the protocols and methods for securely and efficiently transferring data between systems, ensuring compatibility and effective communication across diverse platforms.
| style="color:white; background-color:white; border:solid 1px white; padding: 10px; vertical-align:top;" |[[Layer_-_Data Exchange|Link]]
|}


The key principle is to model for multiple interoperable modular standards in any part of the framework. In this way, industries, organizations and solutions providers can choose the standard per part of the data exchange that suits them best.


===IMDE Standard and versioning===
The challenge in this approach is that the world is big, so there are many standards, for different territories, industries and use cases. Which has resulted in the fact that data exchange is still a technical and organizational nightmare. Example: A Food manufacturer sourcing raw materials and components from multiple suppliers and selling both A-Brand and Private Label products can be dealing with dozens of  different ways to exchange origin and allergen data. While the type of data is 100% the same in all situations: Allergens information has the same data points for raw materials, production materials, semifinished goods and finished goods, and allergen data is not different for A-Brands or Private label brands. But still, from a technical standpoint, organizations need to deal with many ways to export or import allergen related data.
IMDE goal is to support multiple standards and standard versioning (backwards and forwards) for all data points relevant to manufacture, distribute and commercialize discrete products (e.g. food, beverages, fashion, electronics, powertools, adhesives, pet food, personal care, home care, et cetera).


The IMDE framework does not included the Data Exchange layer so that IMDE can be implemented in any DataPool (Like 2BA), Data Network (like GS1-GDSN) or Digital Catalog (like [https://fabdis.fr/ FABDIS] or [https://www.bme.de/ BMECAT]). Data Exchange which can run on the IMDE DataFormat are for example: FTP, RestAPI, SOAP-API, DataPorts. Multiple IMDE DataFormats can be define per DataTopic, so DataTopics can be implemented in Excel, XML and Json based datasets.
The solution is to move away from all-in-one standards, where all standards had the strategy to become THE standard. And where every standard covers all 6 layers: the dictionary, the information model, the message structure, the exchange technology, the [[ID_-_Identifier_Types|identifiers]] and the code lists. This new approach is based on the idea of allowing modular standards with independent solutions for all 6 layers. Also supporting/allowing multiple standard per layer. For example:
* Support data exchange using either an Excel file or an XML message.
* Exchange technology: Indirect via and or more data pools or direct via a RestAPI.


IMDE base principle is to use existing information standards per data topic, like for example:
The key principle is to model for multiple interoperable modular standards in any part of the framework. In this way, industries, organizations and solutions providers can choose the standard per part of the data exchange that suits best.
* [https://www.etim-international.com/ ETIM], the international classification standard for technical products
* [https://www.fao.org/ FAO] for Fish farming and Fishery related information standards
* [https://www.iso.org/ ISO] (e.g. for languages, countries and units)
 
==Data Containers==
 
IMDE supports the exchange of data related to the following entities via so called DataContainers. A data container contains a message or file related to one or more entities of the same type.
 
* '''Material''': Use to exchange data related to physical items, in most cases items Materials can be transported by car, truck, plane or boat. There are multiple typs of materials:
** Raw materials (e.g. eggs, porkbelly, salt, oil, tree trunk)
** Production materials (e.g packaging components like foil & cans and food components like herb-mix or
electronic components)
** Semi-finished goods (e.g. bottled beer without labels, frozen fries not yet packaged), usually produced by the brandowner/product manufacturer).
** Finished goods (e.g. TV, mobile phone, bottle of shampoo, ready to eat salad, smoothie in plastic bottle)
** Handling units (e.g. crates, cases, displays and pallets). Easy check: Materials can be transported by car, truck, plane or boat.
* '''Organisation''': Legal entity like companies, NGOs and government bodies. Easy check: Organisations can be sued in court
* '''Brand''': Covers both product and organizational brands.
* '''Location''': Any place on earth where activities take place, like farms, forests, production facilities, distribution centers and retail stores). Easy check: Locations can be found on Google Maps.
* '''Person''': Individuals/humans like employees and consumers)
* '''Transaction''': Transactions between persons, locations or organisations like orders, invoices, shipments, payments.
 
Every entity will have a defined '''DataContainer'''  within IMDE framework to exchange data related to this entity (e.g. product information for materials)
 
===Data Topics===
The core principles of the IMDE framework are interoperable and modular. That also applies to data messages and files. Within the IMDE framework industry groups will work on defining DataTopics. A DataTopic will contain all datapoints covering a specific topic. Examples are: Packaging Materials, Allergens, Marks or Claims. Any data message can contain one or more DataTopics, depending on the needs in that part of the supply chain. For example
* Packaging Component Supplier will include the Packaging Materials DataTopic
* Herb Mix Supplier will include the Allergen topic
* The Manufacturers that combines that into a consumer product will include both topics in the data message that is send to the retailer.
 
This covers the modular part, the frame becomes interoperable by allowing multiple formats per Datatopic, with defined transformations between the formats. This allows every party both data senders and receivers to work in the format they prefer (no longer deliver multiple formats to all different data receivers).
 
IMDE will support multiple DataFormats per '''DataTopic''' (e.g allegens) and related data points, making sure all industries and all territories can join IMDE. IMDE will also support multiple DataFormats per Data topic. Example: allergen exchange Excel template and Allergen Exchange XSD.
 
* List of [[IMDE Data Topics]]
 
===Identifiers===
Identifiers are essential to accomplish autonomouse communication between machines across the entire value network. But just like with data standards there are also multiple identifiers per entity. That is why IMDE will also support multiple identifiers per entity. Depending on the industry or territory different identifiers can be used to exchange data. Just some examples:
*Material Identifier:
**[https://www.gs1.org/standards/id-keys/gtin GTIN] standard by GS1 for Trade Items
** UMID (Unique Material Identifier, based on [https://en.wikipedia.org/wiki/Universally_unique_identifier UUID])
*Location Identifier:
**[https://true-code.org/ True-code] for identifying locations/facilities.
**[https://www.gs1.org/standards/id-keys/gln GS1-GLN Global Location Number]
*Brand Identifier:
**[https://www.gbin.info GBIN] standard for identifying brands.

Latest revision as of 14:50, 6 October 2024

What is IMDE?

Interoperable Modular Data Exchange (IMDE) is a framework that will enable machine-2-machine exchange data across the entire value network. Minimizing the cost for collecting, using and distributing data. IMDE can overcome a number of the challenges faced by organizations, solution providers and data processors like datapools in terms of improving simplification of data exchange while at the same time improving data quality (consistency, relevance, completeness, accuracy and timeliness). The biggest benefits are in exchanging data with both upstream and downstream business partners. The beneficiaries are machines!, so not humans. The whole framework is designed for fully autonomous communication between machines.

Why Interoperable and Modular?

Many of the initiatives in the past focussed on creating standards for specific use cases, or in specific industries. These standards often covered all six layers of data exchange:

IMDE - 6 Layer Framework for Interoperable Modular Data Exchange
L1 TX Taxonomy Establishes a unified communication language to ensure consistent terminology and understanding across different systems and platforms. Link
L2 IM Information Model Defines the structure and relationships of data elements, providing a common blueprint for organizing and interpreting data. Link
L3 ID Identifiers Supports various standards for uniquely identifying entities such as locations, products, brands and organizations. Link
L4 CL Code Lists Standardizes sets of codes used to represent specific data elements, ensuring consistency and interoperability across different datasets and platforms. Example: Unit of Measure Link
L5 DF Data Formats Accommodates multiple data formatting standards in formats like JSON, XML, or spreadsheet templates, with clear declarations for each data container to facilitate seamless data exchange. Link
L6 DX Data Exchange Defines the protocols and methods for securely and efficiently transferring data between systems, ensuring compatibility and effective communication across diverse platforms. Link


The challenge in this approach is that the world is big, so there are many standards, for different territories, industries and use cases. Which has resulted in the fact that data exchange is still a technical and organizational nightmare. Example: A Food manufacturer sourcing raw materials and components from multiple suppliers and selling both A-Brand and Private Label products can be dealing with dozens of different ways to exchange origin and allergen data. While the type of data is 100% the same in all situations: Allergens information has the same data points for raw materials, production materials, semifinished goods and finished goods, and allergen data is not different for A-Brands or Private label brands. But still, from a technical standpoint, organizations need to deal with many ways to export or import allergen related data.

The solution is to move away from all-in-one standards, where all standards had the strategy to become THE standard. And where every standard covers all 6 layers: the dictionary, the information model, the message structure, the exchange technology, the identifiers and the code lists. This new approach is based on the idea of allowing modular standards with independent solutions for all 6 layers. Also supporting/allowing multiple standard per layer. For example:

  • Support data exchange using either an Excel file or an XML message.
  • Exchange technology: Indirect via and or more data pools or direct via a RestAPI.

The key principle is to model for multiple interoperable modular standards in any part of the framework. In this way, industries, organizations and solutions providers can choose the standard per part of the data exchange that suits best.