Thoughts, Tuning and Model Validation

One day I had the following conversation with my independent model reviewer when he is about to issue a finding to my AML model validation program:

You are saying I don’t have a lock ready for my door, and that poses great safety risks, well, I agree with you, but, please note I haven’t built my house yet, and I don’t even have a door!

That’s a true conversation I had in my career

This is true for any AML compliance program. It takes time for an AML compliance program to mature because it is a kind of cultural transformation, a compliance culture transformation. If we define the five stages of maturity of an AML program, whatever the five stages are, it is, therefore, crucial to know which stage your organization’s compliance program is at so that activities such as assurance, audit, and action items can be conducted in the right context, and ultimately your organizations budget and resources can be spent wisely against the right problems at the right time.

For example, it doesn’t make sense to do a scenario tuning exercise if the AML program hasn’t got any investigation standard setup yet, without which, tuning can take no consistent feedback and the efforts will be spent in vain.

It also doesn’t make sense to focus on delineating what data quality problems exist or focus on “roadmaps” to fix those issues if data ownership and model ownership haven’t been established through a Model Governance Framework.

I can go on and on.

What are the stages of maturity then? One may ask? Compliance Analytics Ltd. defines five stages:

  1. Nascent: no documentation, no framework, and no tuning conducted would be three key characteristics that identify stage one.
  2. Basic: methodologies exist and are documented, and governance and data frameworks are set up, however, the whole governance process and methodologies haven’t been practised and operationalised.
  3. Established: at least one round of tuning, governance, and data quality (DQ) exercise has been operationalized and documented. This must include at least one round of independent assessment being conducted, which could be an independent model review (IMR) or assurance or audit.
  4. Advanced: multiple rounds of tuning, governance, audit, IMR, and DQ have been operationalized. This must include the risk owner’s involvement through risk tolerance, and policy and procedure sign-offs through the governance framework.
  5. Matured: Further, in the advanced stage, new technologies are incorporated through BAU governance and model framework without significant friction and change in policies.

Contact us for an assessment of your maturity stages or design stages suitable for your organization.

Comments

Joseph Bognanno May 27, 2022 at 11:19 pm

I remember that conversation vividly! Glad I was there to hear it.

This demonstrates the value of what I call a “practitioner” versus an “expert”. Sometimes people refer the practitioner as the “in-pert” because they have the “in”side view having lived it.

A big take-away from the above is that there is no “one” framework or approach that is the correct one. There are lots of considerations that need to be made when looking at the appropriate model governance or model risk management framework for a given organization. Right-sizing and maturity are very important concepts to keep in mind when designing, building or assessing whether or not a framework is “fit for purpose”.

Context and facts about the business model, customer base, technology platforms, risks, requirements, etc. etc. are so very important and often overlooked by regulators, and other 3rd parties that try to apply a “one size fits all” or largely academic approach to the subject of good governance and model risk.

Reply

Leave a comment

Your email address will not be published. Required fields are marked *