Securing Autonomous AI Agents: Deterministic Gateways for Tool Use and Function Calling

An AI agent that can call tools turns a persuasive sentence into an action. If the model can send email, query a production database or change a cloud configuration, then anything that steers the model can steer those actions: a poisoned document, a malicious tool result, or an ambiguous request read the wrong way. That is the central problem of AI agent security, and OWASP has a name for it: Excessive Agency.^[1]

This insight is part of my enterprise guide to defending against AI-enabled attacks. It covers the action boundary of the Deterministic AI Boundary Model: whether the model can cause an effect nobody authorised. The principle is the one behind INTERLOCK, my research in progress on controlling high-consequence agent actions: the agent proposes, and an independent gateway decides.

What is excessive agency, and why is it the core agent risk?

Excessive agency is an agent's ability to take damaging actions because it has more functions, permissions or autonomy than its task needs. It is the core risk because it turns every other failure, from prompt injection to a plain model error, into a real-world effect.

OWASP's LLM06:2025 entry names three root causes: excessive functionality (tools the agent does not need), excessive permissions (rights on downstream systems beyond what the task requires) and excessive autonomy (high-impact actions taken without independent checks).^[1] The trigger does not matter: hallucination, weak models and prompt injection end in the same place.

The research shows how wide the gap is once agents get real tools. In GrantBox, Zhang and colleagues connected agents to ten real MCP servers exposing 122 privilege-sensitive tools, then ran 500 injection cases per model against GPT-5, Gemini 3 Flash, Qwen3-Max and DeepSeek-V3, with no defences. Attacks succeeded 91% of the time on average with a ReAct-style agent and 79% with a plan-then-execute agent; the authors call these results preliminary.^[2]

Over-privilege also happens without an attacker. In 544 simulated scenarios where a lower-privilege tool would do, 11 models reached for the more powerful one between 2.6% and 64.9% of the time, and more often after the lower-privilege tool failed.^[3] At ecosystem scale, a census of 12,230 tools on 1,360 MCP servers found that 8.7% of the tools, on 27.2% of the servers, exposed a capability an attacker could exploit.^[4]

Why should the agent never hold its own credentials?

Because whatever sits in the agent's reach can be used or revealed by the agent, and attackers can reach the agent through its inputs. Credentials belong in a gateway outside the model, which acts for the agent only on requests that pass policy, for the current task, with the narrowest scope.

Most agents today are handed a bearer credential: an API key in an environment variable, a token file, a forwarded SSH agent. Jin and colleagues name the structural flaw: the agent can both use and reveal the same credential, so an injection that makes the model paraphrase or send text can do the same to the key.^[5] Their design gives the agent a session-bound handle for one policy-checked intent; a broker holds the secret, makes the call and records the decision. The evaluation is scripted, so treat it as a design pattern, not a measured result.

OWASP agrees: run tools in the user's context with minimal scopes, and enforce authorisation downstream rather than letting the model decide (complete mediation).^[1]

Scope matters as much as custody. MiniScope found six over-privileged ChatGPT and Claude connector configurations; ChatGPT's Gmail connector requested 50 API methods where 32 would do. Asking an LLM to choose the minimum permissions produced the optimal set only 70% to 83% of the time with proprietary models, while computing scopes deterministically from the task added 1% to 6% latency in the authors' tests.^[6]

How strict should tool-call validation be?

Strict enough that the gateway can decide without trusting anything the model says: a closed schema for every tool, typed and bounded arguments, checks against the system of record, and a rule for where each argument may come from. A call the policy does not allow is denied, not repaired.

Progent shows what argument-level policy buys. It enforces rules on which tools may run and which argument values are allowed, in code outside the model. With GPT-4o as the agent, attack success fell from 39.9% to 1.0% on AgentDojo and from 70.3% to 3.9% on Agent Security Bench, even with its approver set to accept every request to widen the policy. Attacks designed to get around Progent raised success only marginally.^[7] The authors are equally clear about the limit: it cannot stop an attack that stays inside the privileges the task genuinely needs.

A sanitised example of a constrained tool and the gateway policy that checks it:

{
  "tool": "issue_refund",
  "schema": {
    "type": "object",
    "additionalProperties": false,
    "required": ["order_id", "amount_aud", "reason_code"],
    "properties": {
      "order_id":    { "type": "string", "pattern": "^ORD-[0-9]{8}$" },
      "amount_aud":  { "type": "number", "exclusiveMinimum": 0, "maximum": 500 },
      "reason_code": { "enum": ["damaged", "late", "not_received", "duplicate_charge"] }
    }
  },
  "policy": {
    "credential": "held by gateway: payments-refunds-only",
    "order_id_from": ["user_turn", "crm_lookup"],
    "require": [
      "order.customer_id == session.customer_id",
      "amount_aud <= order.amount_paid - order.amount_refunded"
    ],
    "budget": { "per_task_aud": 500, "per_day_aud": 5000 },
    "decide": {
      "deny":  ["order.status == 'chargeback'"],
      "ask":   ["amount_aud > 200", "order.flagged_for_review"],
      "allow": ["otherwise"]
    },
    "on_deny": { "tell_agent": "denied", "taint_session": true },
    "log": ["proposed_call", "argument_sources", "rule", "approver", "result"]
  }
}

The closed schema leaves no free-text field for an injection to fill. The require rules check facts against the order system, not the model's claims. order_id_from applies the provenance rule from indirect prompt injection in RAG pipelines. The budget caps the whole task, not each call; cost limits are covered in preventing Denial of Wallet in LLM APIs.

Which actions need a human in the loop, and how do you stop approval fatigue?

Actions that are irreversible, external or above a value threshold, chosen by deterministic policy rather than by the model. Keep approvals rare so each one is read: let policy allow routine actions and deny forbidden ones outright, and show the approver the exact call and where its inputs came from.

Approval before every consequential action is not a control people sustain. Kolluri and colleagues at Microsoft note that frequent approval interruptions lead to confirmation fatigue, in which users stop reading the prompts; they draw this from the human-factors literature rather than measuring it in agents.^[8] On the WASP browser benchmark, a basic agent that asked before every consequential action needed one to nearly four approvals per task, and injected instructions still diverted it in 15% to 61% of cases, depending on the model and site. The approver was all that stood between a diverted agent and the action.

Their system, PRUDENTIA, uses information-flow labels to decide deterministically when approval is needed. On WASP it needed none and no attack got through, but the tasks were simple and the attacks were fixed templates rather than adaptive. On AgentDojo it cut approvals by up to 2.9 times rather than eliminating them.^[8] Progent reports a similar pattern: only 6% of its policy updates needed a person.^[7]

Approval can also be switched off. In the MCP toolchain study, 9 of 10 attack chains built from popular real servers exfiltrated data in at least one of ten trials with the client set to run tools automatically, and the authors argue that confirmation steps are deployment-specific constraints, not reliable defences.^[4] Four rules follow:

1. Policy decides when to ask. Deny, allow and ask are policy outcomes. The model never chooses whether its own action needs approval.
2. The gateway writes the prompt. The approval screen shows the actual call, its arguments and their sources, not a summary the model wrote. Kolluri and colleagues make the same point: show the approver the origin of the data so trusted and untrusted inputs can be told apart.^[8]
3. Approval binds to the call. These arguments, this task, then it expires.
4. Approval is a budget. A rising count of prompts per approver means the policy is doing too little.

My view: a human approval that the model can trigger, describe or route around is a Level 1 control, however solemn the button looks. It becomes part of a Level 2 boundary only when the gateway decides when to ask, writes what the person sees, and refuses to act without the answer.

What do per-call checks miss?

Harm spread across calls that are each allowed, and information that leaks through what the gateway refuses. Per-call policy is necessary but not sufficient: the gateway also needs session state, budgets across the whole task and, for multi-step plans, a check on the plan before any step runs.

Ahad and colleagues showed the first gap in multi-agent pipelines. A single, legitimately phrased request led the orchestrator to split a task into subtasks that were each benign but together violated policy, in 10 of 14 scenarios. Nothing was flagged at input, and every subtask passed the safety classifiers. A check on the whole plan before dispatch caught the violations, though the study is small and its agents were simulated.^[9] MiniScope and Progent state the same limit from the defender's side: neither stops misuse that stays within the granted scope.^{[6, 7]}

The second gap is subtler. In what Chinaei calls causality laundering, an attacker has the agent try a protected action, learns something from the refusal, and leaks that inference through a later call that looks harmless: the read of a protected file is denied, and a permitted email then reports that the file exists. Data-flow tracking misses this because no data flowed from the denied call. The paper shows it in three hand-built scenarios, not at benchmark scale.^[10]

So the gateway has to remember. A denial should tighten what the session can do next (the example policy taints the session). Budgets should cover the task: value moved, records read, recipients. A multi-step plan touching sensitive data should be checked whole before its first step runs. Budgeted control of high-consequence actions is what INTERLOCK investigates; it is research in progress.

How do you log agent actions so you can reconstruct an incident?

Log at the gateway, not in the agent: every proposed call, its arguments and their sources, the rule that decided, any approval, and the result. Give each agent its own identity so every action is attributable, and review the record as whole trajectories rather than single events.

Figuera points out that today's agent observability is structurally compromised: the process writing the activity log is the process being logged, so a compromised or faulty agent can omit, alter or invent its own trace.^[11] His proposal, receipts signed by the receiving services, has no adoption yet, but the principle works today: the gateway, which the agent cannot instruct, writes the record.

Identity matters as much as content. A threat analysis submitted to a NIST request for information notes that when several agents share one API key, the audit log cannot say which agent acted, and that the overhead of separate keys pushes teams towards sharing.^[12] These are modelled threats, not observed incidents, but a log that cannot attribute an action cannot support a finding about it.

Review matters too. On SHADE-Arena sabotage trajectories, in which an agent pursues a hidden goal through individually benign steps, a monitor reading each whole trajectory caught 41% of the malicious runs and a step-by-step monitor caught 58%. TRACE, which accumulates weak evidence across steps, caught 84%.^[13] When an agent's action is disputed, by a customer, a regulator or in court, the questions are who authorised it, on what information, and whether the record is unaltered. For each call, the gateway should record:

• agent identity, user and task;
• the proposed call, with the source of each argument;
• policy version, rule and decision;
• approver, what they were shown, and when;
• the result, including denials fed back to the agent;
• in a hash chain or write-once store, so edits are evident.

Testing against an attacker who adapts to your gateway completes the evidence boundary. The steps are in the AI red-teaming audit checklist for the OWASP LLM Top 10, and for agents that read shared data, the data boundary is covered in tenant isolation in vector databases.

My view: two questions tell a board most of what it needs to know. Which actions can our agents take that nobody approved? And for any action last month, could we prove who allowed it and why? If the answers come from the model's own logs, the boundary is not there yet.