Data Quality? There is NO bad data

All data are trying to tell you something, not seeing it doesn’t mean they are bad data.

In the compliance analytics area, Data Quality is becoming increasingly important. The rise of the CDO (Chief Data Officer) helps to establish a set of data standards. There are many positive things coming out of the data quality practice, however, we do see that sometimes the data quality concept impedes meaningful progress of any meaningful analytics.

The reason is that everyone believes there is good data and bad data, and if data quality is terrible, analytics won’t be effective.

The term Data Quality is largely a distraction, it should be “Data Impact”

The distraction is that when we talk about “quality”, we generally think about good vs bad. To a certain extent, this is true. However, practically, it doesn’t help. Many institutions simply “wait” or are “stuck” at the “fixing the quality” stage. Worse, some of them spend millions and a year to just locate data elements to “fix”. These greatly impede their capabilities of Compliance Analytics, such as AML innovation or AML transformation programs. The view that is fixated on the very definition of “data quality” implies there is a required “correct” data standard, including format, lineage, values, etc. of the said data. Where, in reality, what matters is the impact assessments.

The bloody truth is that there will never be 100% clean data, and we never need 100% clean data. A piece of “bad” from an AML risk perspective might be telling a “good” story of an operational risk! What we need is “impact assessment” capabilities, where we can assess data issues and quantify their impacts on end results as defined by Risk Policies and Risk Appetites.

This must be done through a solid Analytical Framework.

Therefore, here is some advice on DQ:

1. Accept the view that there won’t be 100% “clean” data.
2. Accept the fact that every piece of data is trying to tell you something, “bad” data in particular is shouting at you, risks exist somewhere. So take a broader “risk view” and look closely, don’t simply mark them as DQ issues and look away.
3. Develop and automate impact assessment capability.
4. Last but not least, create analytics through your Model Validation framework that put the principle to work that bad data for one model could be good data for another.

Contact us: haibo@compliance-analytics.co.uk