Data Crossroads has published the Data Management Maturity Assessment Review for the second year in a row. In the first article of the series, we have explained the methodology to measure maturity. Then, we have demonstrated the general trends in data management, data governance, data modeling, and information systems architecture. In this article, we focus on trends in the optimization of data value chains.
In this article, we will:
Explain the structure of this capability
Demonstrate the general trends in the optimization of data chains
Investigate changes in four key performance indicators.
Four components enable this capability. These are process, role, data/deliverable, and tool. Each component consists of a set of items. The full list of items per component can be seen in Figure 1.
Figure 1: Data value chain components in detail.
The logic behind this model is simple. For instance, data lineage describes data chains. Data lineage is also the key deliverable of this capability. A process to develop, document, and maintain data lineage delivers this artifact. Data architects, and database and system engineers take part in this process and document data lineage. An automated data lineage solution assists in documenting data lineage.
Now let us take a look at the general trends in the maturity of data value chain capability.
General trends in the maturity of data value chain capability
In the first article of the series, we have seen the general changes in data management maturity. The maturity of data value chains has slightly worsened in 2020 compared to 2019, as shown in Figure 2.
Figure 2: The comparison of the maturity levels per DM sub-capability.
Some movements between the maturity levels presented in Figure 3 provide possible explanations for this statement.
Figure 3: The changes per maturity level.
In general, this trend seems positive. The number of companies at the two lowest levels of maturity has decreased. Many companies have started initiatives related to the optimization of data chains and the documentation of data lineage. The number of companies that have “capable” status remains at the same level.
The “Orange” maturity scan uses four performance indicators to assess the maturity of this capability. Let us take an in-depth look at each of them.
Indicator 1: “tools to document data value chains/data lineage”
Data chains realize data lifecycles. Data lineage describes data chains. Data lineage is a complex concept. A company can document data lineage using either a manual/descriptive approach or an automated one. In any case, the documentation of data lineage is time and resource intensive. Every method requires the use of an IT tool.
The trends with the usage of tools to document data lineage look controversially, as shown in Figure 4.
Figure 4: Trends in the development of data lineage tools.
From one side, the development has positive trends. For example, the number of companies that do not use any tool has decreased. At the same time, the number of companies that have reached the “capable” status has increased. On another side, the situation with companies at the “in development” and “capable” statuses has worsened.
The knowledge of documented data lineage eases tasks to find and includes new data in data chains.
Indicator 2: “ability to find and include new data”
Information requirements constantly change. To quickly respond to new information needs, a company should promptly find sourcing data and include it in the data processing chain. The trends for this capability look positive, as demonstrated in Figure 5.
Figure 5: Trends in the ability to find and include new data.
The number of respondents at the two lowest levels of maturity has declined. At the same time, the number of companies at the top three levels of maturity has grown.
The ability to find new data works together with the ability to explain the content of data transformation. The last question is also an important indicator of maturity.
Indicator 3: “ability to explain data transformations”
The regulatory and audit authorities very often require the explanation of data transformations. Such explanation is a challenging task. Many business rules and filters define data transformations. Even implemented automated data lineage solutions may not help to get this information. The situation with this capability has seemingly improved in 2020, as shown in Figure 6.
Figure 6. Trends in the ability to explain data transformations.
The number of companies that belong to the first three levels of maturity has decreased. However, more companies have indicated that they have reached “capable” and “effective” levels.
The documentation of data lineage required the coordinated efforts of professionals at different business silos. Therefore, the coordination of data management stakeholders is also an important indicator of maturity.
Indicator 4: “ability to explain data transformations”
Data moves across different business units and business lines. To document data chains, professionals should coordinate their activities across business silos.
The situation with the development of coordination between stakeholders seems to become more positive in 2020, as demonstrated in Figure 7.
Figure 7: Trends in the ability to explain data transformations.
More than 90% of respondents have demonstrated to be at one of the top three maturity levels.
The general level of the maturity of data chain capability has slightly worsened in 2020 compared to 2019.
At the same time, trends remain positive.
The following facts confirm the positive trends:
The number of respondents at the two lowest levels of maturity has significantly declined for each of the four indicators.
The number of companies that have reached the three higher levels of maturities has grown.
To bring this DM sub-capability to a higher maturity level, companies should put additional effort into:
Documenting data and information requirements to prioritize efforts to record data value chains
Applying data lineage documentation methodologies that meet the company’s goals and resource requirements
Properly scoping data lineage initiatives that focus solely on data critical for business operations
In the following article, we will investigate the trends in the maturity of data quality.
Irina is a data management practitioner with more than 10 years of experience. The key areas of her professional expertise are the implementation of data management frameworks and data lineage.
Throughout the years, she has worked for global institutions as well as large- and medium-sized organizations in different sectors, including but not limited to financial institutions, professional services, and IT companies.
Strictly Necessary Cookies
Strictly Necessary Cookie should be enabled at all times so that we can save your preferences for cookie settings.
If you disable this cookie, we will not be able to save your preferences. This means that every time you visit this website you will need to enable or disable cookies again.
3rd Party Cookies
This website uses Google Analytics to collect anonymous information such as the number of visitors to the site, and the most popular pages.
Keeping this cookie enabled helps us to improve our website.
Please enable Strictly Necessary Cookies first so that we can save your preferences!