Harmonizing Data and AI Governance: “To Do or Not to Do”? This is a question every data management professional should ask.
This article summarizes the key topics of my presentation at the upcoming DG&AI Conference in Washington this December.
The original title of my presentation was “Integrating Data and AI Governance.” Several months ago, I believed that data and AI governance frameworks needed to be integrated for a straightforward reason: all components of an AI system—inputs, outputs, and models—are forms of data. Therefore, integration seemed obvious. However, as I delved deeper into the topic, I changed the title to “Harmonizing Data and AI Governance Frameworks” for a specific reason: I realized that factors like AI and data risk-related regulations could influence the integration decision.
The results of a LinkedIn poll I conducted recently also highlighted the differing opinions and experiences on integrating data and AI governance frameworks, as shown in Figure 1.
Figure 1: The LinkedIn poll results on integrating data and AI governance frameworks.
In this article, I will:
- Highlight core factors that impact the harmonization of Data and AI governance frameworks
- Outline the method to develop data, AI, and data risk management frameworks
Five Factors that Impact the Design of a Governance Framework for Data and AI System Management
As demonstrated in Figure 2, I divided the factors into five categories. Let me share some findings and insights for each category.
Figure 2: Factors that influence the decision about integrating data and AI governance frameworks and their design.
(Meta)Data and Data Products
In the industry, we lack a consistent definition of data. In my practice, I use the following definition: “Data is the physical or electronic representation of signals “in a manner suitable for communication, interpretation, or processing by human beings or by automatic means.’”
Why is defining “data” so important? The answer is simple: it helps determine the scope and focus of the governance framework for data management. I identify seven data types, categorized based on factors such as data organization (structured, semi-structured, unstructured), format (digital, non-digital), and origin (authentic or synthetic), among others. This classification also aids in defining the scope of the framework.
Metadata and its management are often overlooked, with few organizations giving adequate attention to this data management capability. Metadata is data that defines and describes other data within a specific context. It possesses several unique characteristics that must be considered. Most importantly, no data handling can occur without metadata, meaning the governance framework must account for this particular data type.
Today, many organizations implement the concept of data products, but definitions and constituent components of data products are inconsistent across the industry. The following components are often considered part of data products: data, metadata, software code/applications, services, hardware and infrastructure, and facilities. This list of potential components illustrates a key challenge for the governance framework: the scope can become quite extensive.
AI definitions, components, and techniques
I reviewed ten different regulatory documents related to AI governance issued by various countries representing five world regions. Comparing these regulations is part of my presentation but is not included in this article. I must admit that approaches are quite different in many views.
However, it’s important to note that no universally aligned definition of an AI system exists. Common features of AI systems include being engineered, machine-based, computational, and capable of processing information. Crucially, the key components of an AI system are input and output data, AI models, and technology platforms. By closely examining the key elements of a data product, one can arrive at the same conclusion I did: an AI system has components similar to those of a data product. This raises the question: wouldn’t it be reasonable to unify the governance framework for data management and AI systems?
Another important point to consider is that different AI models exist, each delivering distinct outcomes, such as decisions, recommendations, predictions, and results for goal-oriented tasks.
Data Management or Governance Frameworks
When developing an internal governance framework, an organization has a choice: it can adopt industry guidelines or develop its own. Several leading industry guidelines are available, including DAMA-DMBOK2 by DAMA International, DCAM®, and CDMC™ by the Enterprise Data Management Council, and the TOGAF® Standard by The Open Group, among others. As I have mentioned in several of my publications, these guidelines offer significantly different perspectives on data management and governance structure. This variation poses another challenge for developing a governance framework: what exactly is governance, and what are its key components?
I use the following approach in my practice: the governance capability oversees data management (DM), as outlined in DAMA-DMBOK2. Governance is one of the DM capabilities that performs three key functions: establishing a framework for DM, governing each DM capability, and coordinating their activities. Therefore, the key components of a governance framework are a DM operating model, an organizational structure, and processes and roles for each DM capability. Furthermore, if the components of an AI system are data, then an AI governance framework should also include these same components.
AI Regulations and Frameworks
AI regulations take different approaches to managing AI systems: risk-based, principle-based, and mixed. Each approach requires a distinct AI governance framework. Frameworks for risk-based AI regulations should be integrated with an enterprise risk management framework, while frameworks for principle-based regulations may or may not include a risk component. The conclusion is clear: we must distinctly differentiate between managing the capabilities of an AI system and managing the risks associated with AI systems.
My analysis of several principle-based data and AI system regulations demonstrated many similarities between these principles. This is also one of the arguments for integrating the governance frameworks for data (management) and AI systems.
Data and AI Risk Management Frameworks
In addition to the previously mentioned reason, there is another reason to include risk assessment in the governance framework for managing data and AI systems—data risks. This topic is particularly familiar to financial institutions that must comply with multiple data-related regulations. Notably, industry guidelines such as DAMA-DMBOK2 and the TOGAF® Standard have already addressed risk-related topics. DAMA-DMBOK2 focuses on risks associated with data and those arising from various data management capabilities. In contrast, the TOGAF® Standard addresses only risks related to the implementation of enterprise architecture.
All of these lead to an inevitable conclusion: alongside a governance framework for data and AI systems management, we must establish a risk management framework to address the risks associated with data management and AI systems.
I hope this review has provided insight into the complexity of the core factors discussed above. However, the key question remains: how can we assess all these factors and arrive at a solution that fits an organization’s needs and resources while ensuring compliance with multiple regulations? I use the revised O.R.A.N.G.E. Data Management Framework (DMF) to answer this question.
Method to Develop Governance Frameworks for Data, AI Systems, and Associated Risks Management
The O.R.A.N.G.E. DMF includes six steps that guide an organization in analyzing all of the factors mentioned above in the logical order, determining a solution for integrating and/or harmonizing different frameworks and designing the resulting framework(s).
Let me briefly outline the key goals and deliverables of each step, illustrated in Figure 3.
Figure 3: The O.R.A.N.G.E. DMF for designing and implementing a governance framework for data and AI system management.
STEP 1: ORGANIZE
The Strategic S.C.O.P.E. Formula helps identify the scope of the required framework. An organization must link the business drivers or strategic goals with the required data management and enabling AI systems. Identifying the scope requires a comprehensive analysis of all the abovementioned factors and additional considerations. This analysis should result in formulating a data management and AI strategy. The deliverables also include selecting an appropriate approach for implementing these strategies.
STEP 2: RENDER
Once the scope is defined, the organization must proceed with designing the frameworks. The Customized Capability D.I.A.G.R.A.M. guides through seven core components that constitute and enable the necessary capabilities. In this case, the focus is on governance capability. The key deliverables should include the required data management and/or AI capabilities, an operating model, organizational structure, processes, roles, and policies for the capabilities in scope.
STEP 3: ACTIVATE
The Integrated Implementation Road M.A.P.S. Method helps select an appropriate method and approach for implementing the designed frameworks and aids in developing an integrated implementation plan.
STEP 4: NAVIGATE
The G.A.I.N. Performance Management Method assists in identifying key performance indicators for data management and AI system capabilities at multiple abstraction levels.
STEP 5: GROW
The G.R.O.W. Maturity Assessment Approach demonstrates how to develop a maturity assessment for various data and AI system management capabilities. It is important to note that measuring performance and maturity are two sides of the same coin. Data Crossroads has developed a comprehensive maturity assessment that integrates both aspects. Those interested in conducting this assessment are welcome to contact me.
STEP 6: EVOLVE
The Transitioning S.C.A.L.E.-Up System outlines principles that should be incorporated into the design of the governance framework, facilitating future scalability and adaptability to new challenges.
I hope I’ve conveyed the core ideas behind the six-hour workshop I’ll be delivering at the DG&AI Governance Conference. Of course, this review is just a drop in the ocean compared to all the topics we will discuss. I look forward to meeting you in Washington if you are interested in this subject.
