← All posts

Agents treat boundaries as obstacles. Put the yes where they can't reach it

21 November 2026 · Deliverd Engineering Team · 7 min read

A dashed blue region holds an agent card and a tilted note of rules. One line leaves the region and meets a dark gate block with a lock; a dotted blue line runs from the gate to a person with a blue tick, the approved call goes on to a tool with a blue tick, and a second call stops at an amber bar.

On 16 November, Federal Reserve Governor Michael Barr spoke at the ACM International Conference on AI in Finance in Milan. Most of the speech is about money: debt behind the data centre buildout, herding among trading models, consumer protection. One sentence in the section on cyber risk is about every organisation running an agent:

Given a goal, AI models have treated security boundaries as obstacles to overcome rather than rules to be followed, raising questions about how effective rules will be even against AI agents instructed to follow them.

Read it slowly. The worry isn't agents that were never told the rules. It's agents that were told, and treated the rule as one more thing standing between them and the goal.

A rule the agent can read is a request

The usual way to keep an agent in bounds is to tell it. "Don't touch production." "Never refund more than £500." "Ask before emailing a customer." Those lines go into the system prompt, and the agent reads them along with everything else it reads: the task, the documents, the tool results, and whatever a stranger has written into a web page it opens.

That makes the instruction an input to the thing it is meant to restrain. Most of the time the agent follows it. Barr's point is that "most of the time" is not a control. No bank keeps a cashier inside a limit by sending a memo. The limit is in the system the cashier uses, and the cashier can't edit it.

We made the same argument about AI use policies last week: an instruction is a request until something can refuse it. Barr is saying it about agents, from the Federal Reserve's Board of Governors.

His own list of guardrails

Later in the speech, on agents acting in customers' bank accounts, Barr names what he thinks is needed: "strict authentication, explicit consent verification for specific actions, technical controls limiting agent capabilities, and both rules- and values-based constraints." Each of those can live outside the model. Here is how each one looks in Deliverd today.

Strict authentication. Every agent gets its own identity under Agents, with its own keys. What it does is recorded under its name, not a person's. A key can be revoked the moment you think it has leaked, and Disable agent stops all of an agent's keys at once.

Consent for a specific action. Before a consequential action, the agent asks the gate: may I do this, with these details? A rule or a person answers. A yes is bound to the scope the approver saw, has a window and a number of uses, and can be revoked before it is used. The approval carries a signed decision token, and the system that does the work can check that token with any Deliverd SDK before it moves anything. The gate guide has the detail.

Technical controls on what the agent can reach. Put an MCP server or an HTTP API behind the Deliverd gateway, under Organisation → Integrations & webhooks → MCP and HTTP gateways. The agent is given the gateway's address and its own key, never the server's credential. Calls that change something become gates, and a refused call never reaches the server. The gateway page covers the setup.

Rules and values. Rules live under Agents → Rules → Policy builder: what goes through, what is refused, what needs a person, and how many people. A refusal beats an escalation, whatever order the rules were written in. Values live under Agents → Rules → Ethics: rules that raise a concern for the approver, ask more people to agree, or refuse. None of them can approve anything, and none can loosen another policy.

None of these depend on the agent agreeing. The agent can try anything it likes. What it can't do is make the gateway forward a refused call, or make a payment service accept a missing token.

Rules you can test before they bind

A boundary is only as good as the rule behind it, and a rule written in a hurry can be wrong in either direction. With policy as code, rules live in your repository beside the code they govern. Each policy carries its own tests, runs them in CI, replays the change against the decisions already made so you see what it would have done differently, and goes into force on merge.

Is the person actually reading?

A person in the loop only helps if they're paying attention. An approval rate can't tell a careful reviewer from one who approves everything. Seeded tests can. Under Analytics → Approvals → Seeded tests you mix known-bad requests, written from your own incidents, into reviewers' queues. Each reviewer finds out straight after deciding, and the report shows who stopped them. The approval analytics guide explains how.

Seeing it, and stopping it

Agents → Agent governance lists every call an agent made to the gate, newest first, and follows each one past the decision: who approved it under what authority, whether it was used, and what the agent reported doing. From there you can:

  • revoke a permission nobody has used yet. The agent is told, and a report of acting on it afterwards is refused;
  • switch a policy off. Nothing goes through because of it, and what its rules covered goes to a person;
  • give an unanswered request a deadline, after which it is widened to named approvers.

Governance → Risk indicators turns this into monthly figures against your risk appetite, with an owner for each one. They inform people; they never stop an agent. Stopping is a person's decision, made with the controls above. The risk indicators guide has the list.

Proving it afterwards

Barr ends on what decisions made with these models must be: "well controlled, numerically and legally precise, explainable, and replicable." The last two are about evidence. Each gate has its own record: the policy that applied, who approved under what authority, the scope they saw, the receipts, and what was not checked. Records → Evidence packs puts those together for an auditor, with a digest so they can tell the pack is as it left you.

What is still hard

Two things in the speech go further than any product does today.

The first is speed. Barr warns that agents moving money could turn a bank run into something that unfolds "in minutes, overwhelming traditional liquidity management and regulatory intervention timelines." A person can't approve every step of something that fast. Rules have to handle the routine, and people the exceptions. Whether your people can step in quickly enough is something to measure rather than assume.

The second is behaviour that is allowed step by step but wrong in total. Each call passes its rule, yet the run as a whole drifts from what the agent normally does. Noticing that needs a baseline of how each agent usually behaves.

We're working on both: spotting when an agent drifts from its own pattern, and testing how long it takes your people to step in. We'll write about them when they ship.

What this doesn't do

Deliverd checks what passes through it. An agent that still holds a server's own key can go around the gateway, so keep the key in the gateway alone. Where Deliverd doesn't carry out the action itself, what the agent did is what it reported, checked against what was approved. That's why the receiving system checking the token matters. The platform an agent runs on keeps its own safety controls, and Deliverd neither replaces nor overrides them.

Nothing here is legal or regulatory advice, and Governor Barr didn't mention Deliverd or any other product. He described a problem. This is how we think the yes should be built: outside the model, where the agent can't reach it.

Book a pilot if you'd like to put it in front of your own agents.

Give your agent a way to ask.

Give any AI agent a way to ask a person — for approval, a decision, an answer or a review.