AI Agent Governance: Security and Audit for Risk Teams

Last updated: September 2026

TL;DR

An agent needs governance a model does not, for three reasons: it holds permissions, it takes actions, and it carries context across steps. A model produces a score that some other system acts on; an agent is the system that acts. Everything genuinely new about governing one follows from that — and the way to unblock a security review is to arrive with the artifact that answers each boundary question, not with a policy document.

What is AI agent governance?

AI agent governance is the set of boundaries and records that make an agent's behaviour legible before it acts and reconstructable after it has acted. Boundaries define what the agent may read, what it may do unaided, and what it must escalate. Records establish what it saw, what it concluded, and who owned the outcome.

If you are reading this, the situation is probably specific: the proof of concept works, the results are good, and the security review has not moved in three weeks. That is rarely because the agent is unsafe. It is usually because nobody has written down the five things a reviewer needs to see, in the form a reviewer needs to see them.

Those five boundaries are the substance of this page: data access, permitted actions, escalation, model boundary, and accountability.

Why governing an agent is not the same as governing a model

Most institutions reading this already run a model risk management framework, and the first useful question is how much of it transfers.

A good deal does. Validation discipline transfers. Documentation habits transfer. Ongoing performance monitoring transfers. Challenger comparison transfers. If you have a mature model risk function, you are not starting from nothing and you should not be told that you are.

What does not transfer is everything that follows from three differences:

• An agent holds permissions. A model has no access rights. It receives features and returns a number. An agent reads systems, and something has to say which ones.

• An agent takes actions. A score cannot close an alert, request a document, or file a report. An agent can, and the set of things it may do without a human is a decision somebody has to make explicitly.

• An agent carries context across steps. A model scores each case independently. An agent accumulates state, which means its behaviour on step four depends on what happened on step one — and reconstructing a decision means reconstructing a sequence, not a single inference.

Nothing in a model risk framework tells you what an agent may do unaided, because a score has no permissions and cannot act. That is the gap governance has to fill, and it is narrower than the "everything is different now" framing suggests.

The five boundaries a review will ask about

Pairing each boundary with the artifact that satisfies it is the part that tends to be missing. A governance overview explains that agents need oversight. A review needs to know which document to put in the folder. The table is the thing to walk in holding.

Where the boundaries actually get set

The boundaries are not a policy exercise conducted after deployment. In practice the sequence runs: map the institution's data and schemas, configure the institution's thresholds and policies, then establish queues and permissions.

That ordering is the argument. Permissions and queues get established as part of standing the agent up — not retrofitted once second line objects. A governance model bolted on afterwards is, reliably, the thing that stalls reviews, because it has to be reverse-engineered from behaviour rather than read off a configuration.

It also answers the question institutions ask most often: can this work with the risk systems we already have? The thresholds and policies are yours. The agent operates inside boundaries the institution configures, which means governance is not something a vendor hands you along with a product — it is something the deployment expresses.

Deployment shape matters here too, because it changes who owns which control. Oscilar's agents can run as a standalone layer on existing infrastructure or as part of the unified platform, and the division of responsibility differs between the two. The integration question is treated properly in adding AI agents without replacing your risk stack.

What the rules actually say right now

This is where most governance writing gestures at "evolving regulation" and moves on. The current position is more specific and more useful than that.

Updated US model risk management guidance explicitly excludes generative and agentic AI from its scope. The OCC's release states that "generative AI and agentic AI models are novel and rapidly evolving. As such, they are not within the scope of this guidance." That is from OCC news release NR 2026-29, issued 17 April 2026, with related guidance in OCC Bulletin 2026-13.

Separately, the OCC, the Federal Reserve Board and the FDIC plan to issue a request for information addressing model risk management generally, and considering banks' use of AI — including generative and agentic AI — in particular. As of this writing it has not been published, there is no comment period, and it is not an RFI specifically about agentic AI.

The interpretation that matters: the carve-out removes a checklist without removing a single one of the examiner's questions. Nobody has told an institution it may stop caring how an agent is tested, monitored, or kept under human oversight. They have declined, for now, to say precisely how. The obligation is intact and the prescribed method is absent, which is a harder position than having a rulebook — and it is why the artifacts in the table above matter more than a compliance mapping would.

SR 11-7 remains the right frame for thinking about this. It is an expectation institutions operate under, and Oscilar's platform is designed to fit it: a full audit trail per decision with stored reasoning, human-in-the-loop by design, and bias and drift monitoring. That is a design intent, not a certification, and the distinction is worth preserving in any conversation with second line.

The audit record, and what it has to contain

"We have an audit trail" is not an answer to a review. The question underneath it is whether the record can reconstruct a decision, and whether a human can read it.

The useful architecture has two layers. There is a raw record, in which every decline reason and every outcome is tied to the data behind it. And there is a derived view built from that record — analyst-readable case cards rather than machine logs.

Two layers because a reviewer asks two questions. Can you prove it? is answered by the raw record. Can your analyst actually read it? is answered by the derived view. Most answers cover only the first, and the second is what determines whether the control is real in daily operation rather than only in principle.

What has to be in the record, per decision:

• The inputs the agent saw

• The reasoning it stored

• The action it took

• The model and version that produced the output

• The human who reviewed, approved, or overrode it

The reason to build it this way is not tidiness. You have to be in a position where whatever decisions were made, you can defend them — and human-in-the-loop is the mechanism that makes the defence possible rather than a feature on a slide. It is worth noticing how far this is from language about agents "automatically handling" work. In a governance conversation, autonomy claims argue against you.

The same structured record is also what later evaluation and tuning read from, since every data point surfaced in a case is tied to its decision. That property is developed properly in how to evaluate an AI risk agent before production.

Who owns the outcome

The roadmap for most stalled agent reviews contains a version of this question, and it is usually the real blocker rather than any technical control.

Accountability attaches to a decision class, not to an individual decision. Nobody signs off on every alert, and no framework requires it. Somebody owns the policy that disposes of a category of alerts, and that ownership is what a reviewer is looking for.

Three roles are worth naming explicitly before the review, because they are the ones that get asked about:

• Who owns the policy — the thresholds and the permitted actions

• Who owns the model — its validation and its ongoing performance

• Who owns the exception — the cases that escalate, and the overrides

The uncomfortable part, said plainly: "the agent decided" is not an answer a reviewer will accept, and it is not one an institution should want. Human-in-the-loop is an ownership design before it is a user-interface feature.

Monitoring an agent in production

The ongoing obligation is real and it is where the carve-out bites hardest — examiners will ask how you know the agent still works, and no rulebook currently specifies the answer.

The capability to point at is monitoring that covers data drift, feature drift and concept drift, with outcome KPIs measured by segment. Segment matters: an agent can hold aggregate performance steady while degrading badly for one population.

The agent-specific complication is that an agent has more surfaces to drift on than a model does. Its inputs can change, its tools can change, and the population it acts on can change, each independently and each with different symptoms. That deserves separate treatment, and it gets it in monitoring drift in AI risk agents.

What to bring to the review

The most useful question anyone has asked us about auditability came from a bank during a proof-of-concept build session. They did not ask whether production decisions were logged. They asked whether audit trails exist for test runs.

The answer is that there are two distinct audit trails. But the question is the interesting part, because it is sharper than what most governance material anticipates. A reviewer does not only want proof of what the agent decided once it was live. They want the record of what you did while testing — because a policy document proves intent, and a test record proves practice. The gap between "we designed this control" and "we exercised this control" is exactly where a review stalls.

So the checklist to walk in with maps back to the five boundaries:

1. Data access — the documented scope and lineage

2. Permitted actions — the explicit action set, with default-deny outside it

3. Escalation — the policy, the thresholds, and evidence of enforcement

4. Model boundary — the per-decision model and version record

5. Accountability — the named owner per decision class

6. Plus the test record — what you ran before go-live, and what it showed

None of that is exotic. It is mostly writing down decisions that were made implicitly during deployment, which is why the institutions that move fastest through review are the ones that established permissions and queues while standing the agent up rather than afterwards.

Governance is not the tax you pay for deploying an agent. It is the thing that makes deploying one survivable.

Frequently asked questions

What is AI agent governance?

AI agent governance is the set of boundaries and records that make an agent's behaviour legible before it acts and reconstructable afterwards. The boundaries cover what the agent may read, what it may do unaided, and what it must escalate; the records establish what it saw, what it concluded, which model produced the output, and who owned the outcome.

How do you govern AI agents in financial services?

By defining five boundaries and holding an artifact for each: a documented data access scope, an explicit permitted-action set with default-deny outside it, an escalation policy with enforced thresholds, a per-decision model and version record, and a named owner per decision class. In a regulated institution these attach to an existing model risk framework rather than replacing it.

Can AI risk agent governance work with an institution's existing risk systems?

Yes, and the boundaries should be the institution's own. Thresholds and policies are configured during deployment rather than inherited from a vendor, and the standard sequence is to map data and schemas, configure thresholds and policies, then establish queues and permissions. Agents can run as a standalone layer on existing infrastructure or as part of a unified platform, which changes who owns which control.

What governance controls are needed for agentic AI?

Data access scope with lineage, an explicit permitted-action set, an escalation policy with thresholds, a model and version record per decision, named accountability by decision class, a two-layer audit record, and ongoing drift monitoring with outcome KPIs by segment.

Does model risk management guidance cover AI agents?

Not currently, in the US. Updated model risk management guidance explicitly excludes generative and agentic AI from its scope — the OCC's release NR 2026-29 of 17 April 2026 states that such models "are not within the scope of this guidance." The OCC, Federal Reserve and FDIC plan to issue a request for information addressing model risk management generally and considering banks' use of AI in particular, but it has not been published and there is no comment period. The examiner's questions about testing, monitoring and human oversight are unaffected.

Who is responsible when an AI agent acts?

A named human, accountable for the decision class rather than for each individual decision. Three ownership roles should be identified before a review: who owns the policy, who owns the model, and who owns the exception. "The agent decided" is not an accepted answer.

How are human review, auditability, and agent performance governed?

Human review is governed by an escalation policy with enforced thresholds and named ownership by decision class. Auditability is governed by a two-layer record — a raw per-decision record tied to its data, and an analyst-readable view derived from it. Performance is governed by monitoring that covers data, feature and concept drift, with outcome KPIs measured by segment.