Data lineage. We regularly post polls on LinkedIn on this topic. We want to collect insights into the experiences and thoughts of our colleagues. We would like to share these insights with you as well. Today we take a look at the types of data lineage.
We got the following results about the usage of different types of data lineage.
1. 9% of those who viewed the poll provided an answer.
2. Almost half (44%) of respondents explore descriptive data lineage.
3. Only 6% of respondents implemented automated data lineage only.
4. The rest use a combination of methods.
We interpret these results as the following:
1. Still, a lot of companies document data lineage manually at a high level of abstraction such as the level of application or conceptual/logical level of a data model.
Descriptive documentation is a highly time- and resource-consuming exercise. A descriptive lineage is also difficult to maintain.
2. On its own, automated data lineage at a physical level is not widely used. Automated data lineage requires a lot of investments and resource at the implementation phase. Maintenance is not an issue as it happens automatically.
3. Many companies use a combined approach.
How do you interpret these results? What types of data lineage do you elaborate on? Please share in the comments section below this post!