Data Quality and Readiness is critical for effective business operations and achieving business goals. Here we talk about “ARM your data” and “PARC your data” methodologies to achieve and maintain Data Readiness.
Here’s a story you have seen played many times… “A company is investing millions of dollars in a project or business initiative. There are huge value and ROI expected from this investment. The project is well planned, and activities get executed. However, once the Testing / Validation phase starts, things start falling apart. And the primary reason is the quality of data. The quality of data could be in the source data itself or the quality of output of any data interfaces, data conversion, or data migration. The project gets delayed, starts running over-budget, there is tension and there is finger-pointing between Business, IT, and the consulting partner/vendor. ”
Setup the foundation – Make Data Management as a separate organization or track
Only a small percentage of organizations explicitly define a data track for a project or business initiative. One reason I commonly hear is that “Data” should be part of IT or technical track because data resides in systems and because systems are owned by IT, data must be owned by IT. While there is an overlap of data and systems, this is an incorrect assumption almost all the time. The simple reason being that ownership of “Data” almost always is and should be the responsibility of the line of business. Or in organizations that have an exclusive “Data Management” teams, it is the responsibility of data is with the Data teams. Data will continue to flow from point of entry or creation to systems of operations, data warehouse, and increasingly external integration.
A separate data track allows for a clear division of responsibility and better accountability.
Methodology for achieving and maintaining Data Readiness
Achieving Data Readiness can be thought of in 2 phases. Phase 1 is reaching a minimum readiness that allows the processes using the data to be minimum viable to use in business operations. It can be considered a project. It has a goal. The phase completes when the goal is achieved. Phase 2 is maintaining and continually improving data readiness that will allow for maximizing the value of data in business operations. It can be considered as an on-going “Quality Improvement” activity.
“ARM your data” is a proven project-based methodology to achieve data readiness.
“PARC your data” is the sister methodology based on “Total Quality Management” that allows for maintaining and continuously improving data quality and readiness.
ARM your data
ARM stands for Assess, Remediate, Manage. It is a proven methodology to achieve a goal for data readiness or quality.
Assess: The first step is the know what your current state of data readiness is. Assessment of data will allow cataloging of current data sources, processes where such data is used, the ownership of data, and the quality of existing data. Assessment should be conducted via a combination of user surveys, reference model mapping, and if a possible real scan of data against basic quality rules. The result of an assessment is a report with observations, the impact of poor data readiness and its impact on business metrics, and clear recommendations and action items for improvement.
Remediate: This is the phase of fixing the gaps identified from the assessment. The activities should include cleansing, standardization, harmonization of data quality. As well as setting up data governance with data quality rules, data policies, data processing workflows.
Manage: Even with a comprehensive assessment and high-quality remediation, data quality is not going to be achieved in a single cycle. It requires iterations to quickly measure the improvement, remaining gaps, and repeating the cycles to close the gaps. The Manage phase of the ARM allows for establishing the monitoring dashboards, KPIs, reports measuring the quality and readiness of data.
Once “ARM your data” has been accomplished, all the assets created during this project should be transitioned to “PARC your data” as part of on-going data governance.
PARC your data
PARC stands for Profile, Audit, Review, Comply
Once data quality and readiness are achieved, we are not done. Data Readiness needs to be continuously managed and improved upon. If left on its own, data quality will deteriorate as new data is created in the enterprise or business conditions change and the expectations of data change. PARC is the closed-loop continuous improvement methodology to achieve on-going management of data quality.
Profile: The first step in a PARC cycle is profiling the latest data. This will help understand what if anything is new in the data or data characteristics. If you use a DQ solution (like DvSum), you can setup alerts for classifying outlier data.
Audit: The second step is to execute the data quality rules. Existing rules that are already setup should be automated so they run automatically. New rules based on new requirements should be created. Data Quality rules can encompass the typical master data rules like Completeness, Validity, Accuracy; As well as process quality rules like Volume of data being processed and cross-system integrity.
Review: This is the third step. Here the exceptions or failures should be reviewed and resolution actions are taken. The resolution could mean fixing of data, or it is processed rule failure, it could mean fixing of an interface. The fixes to the data should be applied to the source system or system of record, so all downstream data can synchronize. If the resolution requires multiple users to collaborate, data quality workflows should be used to streamline to bring efficiency and repeatability to the process.
Comply: The final step in this cycle is to monitor the progress and analyze the data management process itself. Data Stewards and Governance leads should review not only the current state of data but all the history and trends of the data quality metrics to see, how well is the data management process running. How long is it taking to resolve exceptions, how long is it taking to set up new data and take improvement actions to improve the overall process?
Conclusion
In the next set of posts of this series, I will share further details about the inputs, outputs, and processes within each phase of ARM your data and PARC your data.
