Different Types of Metadata

Metadata is data used to define data types. It mainly describes the information of the data itself, including source, size, format or other data characteristics. In database fields, metadata is used to interpret the contents of the database. When data lake insight (DLI) creates a table, it defines metadata, which consists of three columns: column name, type, and column description. In this article, we’ll take a closer look at the different types of metadata and how they are used.

Types of Metadata

1. Classification by Types of Metadata

• Metadata about basic data: Basic data refers to all data managed by data sources, data warehouses, data marts, and applications. Metadata of basic data includes definition, all description of structure.
• Metadata for data processing: Information on data loading, update processing, analytical processing, and management, such as data extraction, data transformation, and aggregation rules.
• Metadata about the organizational structure of the enterprise: It specifically includes management data and information related to the enterprise, such as user access to data warehouses, data sources, and data marts’ permission information.

2. Classification by Abstraction Level

• Concept-level metadata: It includes the full description of the business, such as defining main business entities, characteristics and their interrelationships, system usage, defined queries, views, and existing applications.
• Logic-level metadata: It includes the relational scheme of the database, the logical multi-dimensional model, etc., and the rules for data extraction/transformation are generally described in pseudocode or mathematical language.
• Physical-level metadata: This includes SQL code corresponding to business rules, index files for relationships, and code for analytical applications.

3. Classification by Tasks Undertaken by Metadata

According to the task classification of metadata in the data warehouse, it can be divided into static metadata and dynamic metadata.

Static metadata is mainly related to data structures and includes the following elements:

• Name-class: a symbol used to differentiate data when providing identification to the system.
• Description-class: Describes various data elements in the data warehouse.
• Format-class: Provides the expression rules of data in the data warehouse.
• Data type: The type of data in the data warehouse.
• Relationship-class: Describes the relationship between various data objects in the data warehouse. For example, there is a buying relationship between a customer and an item.
• Domain-class: Used to describe the valid range of values ​​for data in the data warehouse.
• Business-rule-class: Used to describe the rules that the data in the data warehouse must abide by in business processing. For example, Customer_ID represents the customer’s number, starting with A for collective customers and B for individual customers.

Dynamic metadata is mainly related to the state and usage of data, including the following elements:

• Data quality table: Used to describe the accuracy, completeness, consistency, and validity of the data in the data warehouse.
• Statistical information: Count the users who access the data, the access time and the number of accesses. These statistics have high reference value for the improvement of data warehouse performance.
• Status-class: Used to track the health of the data warehouse. For example, the last backup time of the data, the time required for the backup, the error conditions, etc. The running status of these systems helps data warehouse managers understand the performance of the data warehouse.
• Processing-class: Used to describe the usage and management characteristics of the data warehouse system. For example, how to use the data, a general formula that summarizes the data, etc.

4. Classification from the User’s Point of View

This classification method is related to the purpose for which the metadata is used. From the user’s point of view, this classification method does not have a unified standard, and in practice, it is often divided according to the principles of habit, feasibility and application.

It is generally divided into two categories: technical metadata and business metadata. Technical metadata is about the development, maintenance and management of all the analysis, design, development, management and other metadata closely related to technology in the information technology environment. It is the technical link connecting development tools, applications and systems. Business metadata makes the services of an enterprise environment more understandable to end users, providing easy browsing, navigation, and data querying for the interpretation of business goals and processes.

Technical metadata includes description data for databases, data warehouses, data dictionaries of data marts, codes of data conversion rules, and the like. E.g:

• Descriptive information about the architecture and scheme of the operating system, data warehouse and OLAP system, table structure, attribute restrictions, view information, etc.
• A description of the physical and implementation-level mappings and dependencies between different information sources, data warehouses, and OLAP systems.
• Extract scheduling metadata, including extracting the complex relationship of scheduling, the mapping between source data and target data.
• Metadata for time synchronization.
• User permission information.

Business metadata includes application-specific documentation (user profiles, storage maps, usage tips, navigation aids, etc.), business concepts and terms, details of predefined queries and reports, contextual information, weights and measures descriptions, date formats, dictionaries, subject headings table etc. E.g:

• Business concept model.
• Multidimensional data model, aggregation principles of dimensions, dimension categories, data cubes, data marts.
• Dependencies between business conceptual models and physical models.
• Supports browsing and navigation for business concepts.
• Support browsing and navigation oriented to business concepts.
• Support dynamic immediate query and data mining.

Conclusion

Thank you for reading our article and we hope it can help you to have a better understanding the different types of metadata and how they are used. If you want to learn more about types of metadata, we would like to advise you to visit Gudu SQLFlow for more information.

As one of the best data lineage tools available on the market today, Gudu SQLFlow can not only analyze SQL script files, obtain data lineage, and perform visual display, but also allow users to provide data lineage in CSV format and perform visual display. (Published by Ryan on Jun 26, 2022)