Data Crossroads has published the Data Management Maturity Assessment Review for the second year in a row. In the first article of the series, we have explained the methodology to measure maturity. Then, we have demonstrated the general trends in data management, data governance, data modeling, and information systems architecture. In this article, we focus on trends in the optimization of data value chains.

In this article, we will:

  • Explain the structure of this capability
  • Demonstrate the general trends in the optimization of data chains
  • Investigate changes in four key performance indicators.

Data value chains

In the “Orange” model of data management, a data chain is a business capability that defines the set of actions to transform raw data into meaningful information. Data chains deliver the business values of data management.

Four components enable this capability. These are process, role, data/deliverable, and tool. Each component consists of a set of items. The full list of items per component can be seen in Figure 1.

Data value chain components in detail.

Figure 1: Data value chain components in detail.

The logic behind this model is simple. For instance, data lineage describes data chains. Data lineage is also the key deliverable of this capability. A process to develop, document, and maintain data lineage delivers this artifact. Data architects, and database and system engineers take part in this process and document data lineage. An automated data lineage solution assists in documenting data lineage.

Now let us take a look at the general trends in the maturity of data value chain capability.

General trends in the maturity of data value chain capability

In the first article of the series, we have seen the general changes in data management maturity. The maturity of data value chains has slightly worsened in 2020 compared to 2019, as shown in Figure 2.

The comparison of the maturity levels per DM sub-capability.

Figure 2: The comparison of the maturity levels per DM sub-capability.

Some movements between the maturity levels presented in Figure 3 provide possible explanations for this statement.

The changes per maturity level.

Figure 3: The changes per maturity level.

In general, this trend seems positive. The number of companies at the two lowest levels of maturity has decreased. Many companies have started initiatives related to the optimization of data chains and the documentation of data lineage. The number of companies that have “capable” status remains at the same level.

The “Orange” maturity scan uses four performance indicators to assess the maturity of this capability. Let us take an in-depth look at each of them.

Indicator 1: “tools to document data value chains/data lineage”

Data chains realize data lifecycles. Data lineage describes data chains. Data lineage is a complex concept. A company can document data lineage using either a manual/descriptive approach or an automated one. In any case, the documentation of data lineage is time and resource intensive. Every method requires the use of an IT tool.

The trends with the usage of tools to document data lineage look controversially, as shown in Figure 4.

Trends in the development of data lineage tools.

Figure 4: Trends in the development of data lineage tools.

From one side, the development has positive trends. For example, the number of companies that do not use any tool has decreased. At the same time, the number of companies that have reached the “capable” status has increased. On another side, the situation with companies at the “in development” and “capable” statuses has worsened.

The knowledge of documented data lineage eases tasks to find and includes new data in data chains.

Indicator 2: “ability to find and include new data”

Information requirements constantly change. To quickly respond to new information needs, a company should promptly find sourcing data and include it in the data processing chain. The trends for this capability look positive, as demonstrated in Figure 5.

Trends in the ability to find and include new data.

Figure 5: Trends in the ability to find and include new data.

The number of respondents at the two lowest levels of maturity has declined. At the same time, the number of companies at the top three levels of maturity has grown.

The ability to find new data works together with the ability to explain the content of data transformation. The last question is also an important indicator of maturity.

Indicator 3: “ability to explain data transformations”

The regulatory and audit authorities very often require the explanation of data transformations. Such explanation is a challenging task. Many business rules and filters define data transformations. Even implemented automated data lineage solutions may not help to get this information. The situation with this capability has seemingly improved in 2020, as shown in Figure 6.

Trends in the ability to explain data transformations.

Figure 6. Trends in the ability to explain data transformations.

The number of companies that belong to the first three levels of maturity has decreased. However, more companies have indicated that they have reached “capable” and “effective” levels.

The documentation of data lineage required the coordinated efforts of professionals at different business silos. Therefore, the coordination of data management stakeholders is also an important indicator of maturity.

Indicator 4: “ability to explain data transformations”

Data moves across different business units and business lines. To document data chains, professionals should coordinate their activities across business silos.

The situation with the development of coordination between stakeholders seems to become more positive in 2020, as demonstrated in Figure 7.

Trends in the ability to explain data transformations.

Figure 7: Trends in the ability to explain data transformations.

More than 90% of respondents have demonstrated to be at one of the top three maturity levels.

Conclusion

  • The general level of the maturity of data chain capability has slightly worsened in 2020 compared to 2019.
  • At the same time, trends remain positive.
  • The following facts confirm the positive trends:
    • The number of respondents at the two lowest levels of maturity has significantly declined for each of the four indicators.
    • The number of companies that have reached the three higher levels of maturities has grown.

Development tips

To bring this DM sub-capability to a higher maturity level, companies should put additional effort into:

  • Documenting data and information requirements to prioritize efforts to record data value chains
  • Applying data lineage documentation methodologies that meet the company’s goals and resource requirements
  • Properly scoping data lineage initiatives that focus solely on data critical for business operations

In the following article, we will investigate the trends in the maturity of data quality.

Compare the maturity status of your company by performing a Data Management Maturity Scan.