When a Bank Does Not Build the AI: Governing Third-Party Models and Agents
Artificial intelligence is
becoming embedded in banking technology, but banks are not developing or
directly operating all of it. Financial institutions increasingly rely on
providers whose platforms incorporate predictive models, generative AI, large
language models, and agentic capabilities into lending, fraud detection,
customer interactions, document processing, and other business processes.These technologies do not present identical risks. A
predictive model that produces a fraud score behaves differently from an LLM
that summarizes a case or an AI agent that can retrieve information, invoke
tools, and initiate actions. The governing question is therefore not simply
whether a product “uses AI.” Banks need to understand what the technology can
access, what it can produce or influence, what actions it can take, and how
those activities could affect customers and banking operations.
A bank may not control how a provider develops its
technology, but it remains responsible for determining whether that technology
is appropriate for its intended use. Effective governance requires sufficient
visibility, accountability, control, and evidence to manage the resulting risk.
Start With Visibility, but Go Beyond a Product
Inventory
AI functionality may be embedded within a larger platform
or workflow rather than introduced as a standalone application. An inventory is
an important starting point, but a list of vendors and use cases is not enough.
For each capability, the institution should understand:
- Its intended business purpose and accountable owner
- The type of model or AI functionality involved
- The data it can access and how that data may be used
- The decisions, recommendations, or actions it can influence
- The degree of autonomy granted to the system
- The role of human review, approval, and override
- The metrics used to evaluate performance
- The evidence retained for monitoring, investigation, and audit
- How material changes are communicated and approved
This information allows the bank to identify not only where
AI exists, but where it creates meaningful customer, compliance, operational,
or reputational exposure.
Govern According to Risk and Operational
Behavior
Third-party AI should not be treated as a single risk
category. Oversight should be proportionate to the sensitivity of the data, the
materiality of the use case, the potential customer impact, the system’s
autonomy, and the institution’s ability to detect and reverse an undesirable
outcome.
A tool that helps an employee summarize internal
information does not require the same controls as one that contributes to a
lending decision. Likewise, an agent that gathers information and recommends an
action presents a different risk from one authorized to execute that action.
Banks should ask practical questions:
- Can the system create customer-facing content?
- Can it influence a consequential decision?
- Can it access confidential or regulated information?
- Can it invoke tools, update records, or initiate transactions?
- Must a qualified person approve the result?
- Can an action be stopped or reversed?
- Can the institution reconstruct what occurred?
Existing model-risk, compliance, information-security, and
third-party risk disciplines provide an important foundation. They should not,
however, be treated as a complete answer for every generative or agentic
system.
Expand Vendor Due Diligence
Traditional vendor due diligence remains necessary, but
AI-enabled products introduce additional questions.
Banks should understand what role the AI performs, what
information supports its outputs, how performance is evaluated, and whether the
provider depends on outside model, cloud, or data services. Due diligence
should also address:
- Data retention, isolation, and deletion
- Whether bank or customer data may be used for model training
- Known limitations and prohibited uses
- Human-review and override controls
- Access permissions and tool-use restrictions
- Material model, data, or provider changes
- Testing for unsupported or unintended output
- Incident detection, notification, and investigation
- Business continuity, portability, and exit planning
Banks do not need access to every element of a provider’s
proprietary technology. They do need enough operational transparency to
evaluate its role, manage its risks, and fulfill their own responsibilities.
Require Evidence, Not Just Assurance
Oversight cannot end when a contract is signed or a product
is implemented. Models, data sources, configurations, and underlying services
can change over time.
Depending on the use case, ongoing evidence may include
performance results, exception patterns, human overrides, model or service
versions, incidents, and actions initiated by AI agents. For agentic systems,
appropriate traceability may also include the instructions provided, evidence
retrieved, tools invoked, approvals received, and resulting actions.
Governance should emphasize enterprise-managed AI, controls
based on data classification, human accountability for outputs, and additional
safeguards for systems that execute multistep tasks or interact with external
services. Banks should consider an internal benchmarking approach based on...