As organizations marshal their data to drive business value throughout the enterprise, it’s becoming more important than ever to ensure the quality of this data. Building data quality, data validation testing, and aligning the data with your organization’s master data (such as products, customers, assets, and locations) is critically important to ensure your reporting is consistent and accurate.
Whether you’re building the latest container application or trying to build a data warehouse, you need to ensure your data is consistent and of high quality. Although data quality isn’t the flashiest technology topic, it’s one of the most important for any data-driven organization.
What Is Master Data Management?
Master data management (MDM) isn’t just an option on your SQL Server installation process. The practice of MDM is designed to ensure there’s a single source of truth for important business data such as customers, products, and locations.
The general concept around MDM is to build a trusted master data store other systems can query to ensure a common source of truth. The master data store will have all the data at a detailed level, and different systems may use different levels of detail. For example, the finance organization may care about the cost of goods sold, whereas the shipping department may be more concerned with how many cases of the product can fit on a palette.
MDM projects need to be driven by the business and not the IT organization. First, the IT team likely lacks the specific domain knowledge to grasp the nuances of the business data. Second, these projects tend to be costly and go beyond the scope of simple technology resources.
Though these projects require investment, the payoff is worth it, as they can deliver exceptional business value. For instance, if you think of the impact of an improperly calculated margin over thousands or millions of units, the cost can be tremendous.
What Is Data Governance?
MDM is one part of the solution and works in concert with data governance. Data governance is the process of managing the integrity, security, usability, and availability of data in your enterprise systems based on the standards and policies of your organization. The importance of good governance processes can’t be overstated.
For instance, simply cataloging your data—the first step in data governance—can protect your organization in the event of a data breach and make reporting projects much easier, since you know where your mission critical data resides. Many tools can catalog data, but without the basics of data governance in place, these tools provide far less value.
Your organization needs to value its data and dedicate resources to focus on data governance. This is typically in the format of a dedicated governance team, whose role is to ensure data sets meet the business goals of the data governance project. These business outcomes include the following:
- Avoiding inconsistent data silos in different departments
- Agreeing on common data definitions to allow for shared understanding of data
- Complying with data privacy laws and other regulations
Breaking down these data silos is challenging without a dedicated enterprise architecture organization. Building such a team requires a diverse organization. Your data quality teams should be staffed with both IT and business professionals who are familiar with the business data and common data structures. Both data quality and MDM have similar goals, and the projects need to work hand in hand to ensure consistent, secure data.
Bringing MDM and Data Governance Together
These two processes must align for business goals: MDM is severely compromised if you’re unsure of the accuracy of the data or if it can be changed without a process. Data governance places data stewards across the organization to ensure data changes through a predefined process.
The next steps of maturity introduce dispute resolution and escalation processes to address inevitable disputes between departments. It’s important for your data stewards to understand both the central goals of your master data management process and the business needs of the organization, which may resist change. Investments in these people will go a long way toward fulfilling your data quality goals. These projects need to be coordinated to deliver high-quality business intelligence to your organization.
Choosing an MDM Solution Suited to Your Strategy
There are several factors to consider when choosing a master data management solution for your organization. You should evaluate the capabilities of the tools; some tools may have overlapping capabilities or integrations with your ETL process. For example, some MDM tools will allow transform, normalized, and schema mapping. Though some overlap with your ETL solution is expected, you want to ensure you aren’t duplicating your tools. Some other beneficial features include automated approval workflows, the ability for subject matter experts to approve changes to customer master data, and ensuring updates are complete, correct, and error-free.
The format of master data can sometimes align to graph structures—think of hierarchies such as continent, country, state/province, postal districts, all the way down to streets. These data structures can be supported in traditional MDM tools. In custom development, however, they may be built into a graph database. A graph database consists of nodes (things) and edges (connections), and it can do an excellent job of representing hierarchies, a data structure which can be hard to model in a relational database.
Consolidation and Data Integration
There are several goals to a master data management project—one of which is consolidating several data sources into a centralized hub acting as the source of truth. One major issue with many business intelligence projects is their propensity to have long time overruns. This isn’t because of technical challenges but because of the sheer number of people and groups involved in the project. Anything capable of reducing the overhead by assisting or shortening the workflow can help reduce the overall length of the project. One of these things is integrating your master data processing into your downstream operational systems.
Using a consolidated approach to integrating your master data can reduce these timelines by simplifying your data flow. In this style of data model, updates to master records are made to the central repository and pushed out to the child systems. This style of implementation is faster, and it provides for an easier data governance process. Beyond data governance, having a consolidated hub can allow you to quickly build a centralized reporting capacity, providing a single source of truth for your application.
Though conceptually having a data hub and pushing master data to source systems seems simple, it can involve a fairly complex ETL process. Each of your core source systems may have different APIs or data structures requiring some degree of customization. Some enterprise ERP systems like SAP may combine the master data management solution with their data integration solution. But even in an organization with a monolithic system, you’ll likely have some subset of independent systems requiring customization for data integration.
Getting Started With MDM
Though you’ve learned a lot about the technologies and implementation patterns around master data management, the technology is probably the least important piece of your MDM effort. Getting your organization on board with a data-centric culture is an important step. This doesn’t just mean your executive leadership should invest in software—instead, it means they should invest in data teams who can support your data governance and master data management efforts. Your team leads also need to understand their business data processes may need to change to reflect those efforts.
Beyond MDM efforts, the next step to elevating your data culture is to start managing your reference data. Reference data refers to a subset of your master data used across your organizations. This data may be private or public—some examples include an ISO list of country codes or transaction codes or private risk management data, such as actuarial tables for different disasters. Just like product lists, having consistent reference data across your environment allows for more consistent records and better overall data quality. Without having centralized management of this data, you can have different departments using different standards, which can result in silos.
If you’re getting started with MDM or data quality assurance, it can be daunting to bring different parts of your organization together. Sometimes, you need guidance to help you better understand your company’s processes and data flows.
Consider SolarWinds® Task Factory® when you’re looking for this guidance, as it can make a big difference in your operations. Get started with a free trial of Task Factory and see for yourself what it can do for you and your organization.
Joey D’Antoni is an ActualTech media contributor and a principal consultant at Denny Cherry and Associates, Microsoft Data Platform MVP, and VMware vExpert.
