For financial institutions, the concept of model validation is not necessarily new. As soon as banks and credit unions began automating certain loan decisioning and back office processes, it became necessary to conduct testing to compare the output of that automation against real-world scenarios to ensure their accuracy and effectiveness. At a certain point, regulators formalized requirements for model validation and reporting and the industry has continued along that path for years.
What is different today is the introduction of much more sophisticated machine learning and artificial intelligence (AI) capabilities into the business of banking. Today, AI-enabled functionality is employed throughout an institution – from the teller station within the branch to back office loan decisioning and pricing strategies. And while virtually every financial institution claims to leverage these technologies in some capacity, the reality is that the degree to which each is utilizing it varies widely – something that makes true evaluation of level of risk a difficult task in today’s environment.
As development and market adoption of machine learning tools has increased, so too has regulatory scrutiny. Most recently, the FDIC (Federal Deposit Insurance Corporation) issued a notice and request for information (RFI)on“Standard Setting and Voluntary Certification for Models and Third-Party Providers of Technology and Other Services.”Its main purpose is to reviewwhether the industry can create a set of standards forinnovativetechnologies like artificial intelligence and machine learningrelated to financial services, and if the third parties creating these programs could self-regulate through voluntary certification.
The FDIC’s request isrelated to its FDiTech initiative, whichfocuses on ways to promoteand increase theeffective adoption of innovative technologiesfor FDIC-supervised financial institutionswithout increasing the costs or regulatory burden. As mentioned, this recent RFI seeks public feedbackfrom outside experts on two areas – whether a standard-setting and voluntary-certification program could be created that wouldhelp support financial institutions as they implement these new technology models and manage their risks; and if a program could be established to conduct due diligence of thethird-parties providingthese technologies and services through certifying and/or assessing the conditions of their operations. Furthermore, the FDIC wants to specifically focus on those technologies and services developed and provided by fintechs. This move underscoreswhy it is importantfor both financial institutions and fintechs to be aware of these issues. Ultimately, they will both need to be able to show “transparency” (meaning the ability for a reviewer to assess a model’s structure, equations, data and assumptions used) and “explainability” (meaning being able to understand whya model chose a particular decision or outcome) from a regulatory standpoint when it comes to model validation reporting. While this is only an RFI now, it could lead to bigger, more concrete changes and regulations down the road.
Financial institutions already recognize the operational efficiency and cost benefits of implementing these systems but now, they must also clearly understand how the models actually work and the potential reputational risk that they may present.
Customers and members trust their banks and credit unions to make fair, equal decisions on loans and other banking services. This is the basis on which our banking system exists. If, however, a machine learning algorithm begins to create different lending criteria based on a specific zip code, for example, this can unleash a litany of problems, both regulatory, legal and reputational, for the institution. To avoid this, FIs must adopt a “trust, but verify” approach to model validation and evaluation of their AI and machine learning partners.
This begins with ongoing bias testing, which can be thought of as “AI Quality Control,” to monitor for biases that might develop over time as the AI continues its machine learning processes. Financial institutions should pair this with regression testing as well to help ensure that the AI is not learning “bad habits” as it continues to evolve its algorithm. By evaluating the output of the system on a regular basis, institutions can more readily identify any rejection anomalies or variances on a certain field (i.e., zip code, sex and/or name), address those more quickly, and report steps taken back to regulators.
Fortunately, there are some existing models that provide examples to the approach that institutions can take. Model validation for machine learning is not dissimilar from FIs’ evaluation of data streams in compliance with Bank Secrecy Act/Anti Money Laundering requirements, for example, or the steps taken to comply with UDAAP (Unfair, Deceptive, or Abusive Acts or Practices) for card-issuing institutions.
While bias testing isan important step, it is one of many needed for creating atruly accurate, effective process. Model validation and testing should also encompassseveral other keyaspects, digging into and weighing all facets of the model itself. This includesreviewing the model’s concept and its performance–scrutinizing its assumptions, how its features are selected and distributed, and how accurately it performs and handles missteps or errors (and if it can adjust accordingly). The model’s data processing should also be considered, reviewing everything from its data’s quality and how its managed and stored, to how it may integrate with other areas or systems.Its reporting and governance capabilities should also be included to confirmthe model’s error reportingfunctions and its controlsrelated to data maintenance. Finally, it is critical tovalidate the model’s overall security, specifically its policies and controls for keeping assets confidential and the risk management related to it.
Ultimately, it is financial institutions that have the most to gain – and to lose – if bias within the algorithm rears its head. Just as Apple recently learned with claims of sexism tied to decisioning for its Apple Card, it is not enough to blame the AI because in the eyes of customers and members, the institution owns the relationship and as a result, carries the weight of reputational risk if something goes wrong. In addition to developing a greater understanding of the algorithms used and exactly how the AI’s machine learning works, financial institutions would be well advised to go a step further and carefully review the language within their fintech partner contracts, particularly as it relates to indemnification clauses.
If the last few years have shown us anything regarding machine learning, it is that the pace of innovation is accelerating at a level that our industry has not seen before. While this presents exciting opportunities for financial institutions and their customers and members, it also requires greater levels of scrutiny and validation to ensure that strides made in leveraging AI and data analytics to provide customized offers and services to strengthen relationships are executed in a way that protects the integrity of the institution and its brand.
Mike Morris, CISA, CISSP, is a partner at Wipfli LLP, a leading national accounting and consulting firm serving clients across a diverse spectrum of industries, including financial institutions, services and technologies.