Agents with too much power

In AI that takes actions, an assistant read a restaurant’s web page and tried to pay a deposit nobody asked for. In Inside an agent harness, a coding agent read a project doc and tried to upload a secrets file. In both, the model was steered by text it read, which the prompt injection lesson explains, and in Poisoned retrieval and memory, by text it retrieved or remembered.

No current model is reliably immune to that. So the question this lesson asks is a different one: when an agent is steered, or simply makes a mistake, how much can it do? The answer is set by the people who build and configure it, through its tools, its credentials, where it runs, and when it has to ask.

Where this sits on the map

LLM03:2026 is from OWASP’s Top 10 for LLM Applications 2026 (it was LLM06:2025); the ASI entries are from the Top 10 for Agentic Applications 2026. The ATLAS evidence levels mean what they did in the last lesson: Demonstrated on a realistic system, Realized in a real incident (ATLAS data, release 2026.09). NIST’s AI 100-2 E2025 is more cautious about agents in general, noting that “security research focused specifically on agents is still in its early stages.”

Excessive agency

OWASP defines excessive agencyExcessive agencyAn AI system able to do more damage than its job needs, through tools it does not use, permissions wider than its task, or high-impact actions taken without a check.Open in glossary as “the vulnerability that enables damaging actions to be performed in response to unexpected, ambiguous or manipulated outputs from an LLM, regardless of what is causing the LLM to malfunction.” The last clause matters. A prompt injection is one cause; a model’s plain mistake is another, as the wrong email to the wrong Sam was in the approve-or-deny lab. The fix does not depend on which.

It names three root causes:

• Excessive functionality. The agent has tools its job does not use, such as a mail tool that can send when the job is only to read.
• Excessive permissions. A tool connects with more rights than it needs, such as database credentials that can delete when the job only reads, or one shared account that can see every user’s files.
• Excessive autonomy. High-impact actions run without anyone checking them.

A documented case shows all three at work. In May 2025, Invariant Labs showed an agent with GitHub’s MCP server (the Model Context Protocol connects tools to agents) asked to look at the open issues in a public repository. One issue, written by an attacker, steered it into reading the user’s private repositories and publishing what it found in a pull request on the public one (Invariant Labs). The authors stressed that this was “not a flaw in the GitHub MCP server code itself”: the agent did what its token allowed. They also noted that the client asks before tool calls by default, but many users choose “Always Allow”. Their main recommendation was narrower permissions, such as limiting an agent to one repository per session.

Try it

The lab has four scripted agents, each steered toward one harmful call by a planted instruction (named, not written out). No model runs; the steps are fixed, and only the program around the agent changes. You set four kinds of control:

• Tools: turn off a tool the agent could call.
• Credentials: read only, or read and write.
• Sandbox: block outbound network to destinations not on an allowed list.
• Approvals: never ask, ask before anything that changes or sends, or ask before everything.

# Each tool call is checked by the program around the model, never by the model.
# Risk: "read" looks only, "change" alters something, "external" sends something out.
TOOLS = {"read_email": "read", "send_email": "external",
         "read_notes": "read", "post_message": "external"}

def gate(tool, config):
    risk = TOOLS[tool]
    if tool not in config["tools"]:
        return "stopped: tool not given"
    if risk != "read" and config["scope"] == "read":
        return "stopped: read-only credentials"
    if risk == "external" and config["sandbox"]:
        return "stopped: no outbound network"
    if config["ask"] == "changes" and risk != "read":
        return "asks you"
    return "runs"

def run(name, steps, config):
    print(name)
    for tool, what, kind in steps:
        print(f"  {gate(tool, config):30} {what} ({kind})")

# The steps are scripted; "injected" marks the one an attacker's text asked for.
inbox = [("read_email", "read new emails", "the job"),
         ("send_email", "forward private emails outside", "injected")]
research = [("read_notes", "read private notes", "the job"),
            ("post_message", "post the notes publicly", "injected"),
            ("post_message", "post the answer", "the job")]

everything = {"tools": set(TOOLS), "scope": "write", "sandbox": False, "ask": "never"}
run("Inbox, every tool, no checks:", inbox, everything)
run("Inbox, no send tool:", inbox, {**everything, "tools": {"read_email"}})
run("Research, sandboxed:", research, {**everything, "sandbox": True})
run("Research, ask before changes:", research, {**everything, "ask": "changes"})

Inbox, every tool, no checks:
  runs                           read new emails (the job)
  runs                           forward private emails outside (injected)
Inbox, no send tool:
  runs                           read new emails (the job)
  stopped: tool not given        forward private emails outside (injected)
Research, sandboxed:
  runs                           read private notes (the job)
  stopped: no outbound network   post the notes publicly (injected)
  stopped: no outbound network   post the answer (the job)
Research, ask before changes:
  runs                           read private notes (the job)
  asks you                       post the notes publicly (injected)
  asks you                       post the answer (the job)

What you saw

Removing what the job does not use is free. The inbox summarizer never needed to send, so turning that off cost nothing. This is OWASP’s first two fixes, “Minimize tools” and “Minimize tool functionality”, and the OWASP agentic list adds a name for the general idea, least agency: “avoid unnecessary autonomy”. The harder part in practice is knowing in advance what a job will need.

Narrow credentials, in the user’s context. Least privilegeLeast privilegeGiving a program, person, or agent only the access its task needs, so a mistake or a takeover can do less harm. OWASP's agentic list extends the idea to autonomy as least agency.Open in glossary means a tool connects with only the rights its job needs, ideally as the user on whose behalf it acts. OWASP’s example is an email tool authorized with a read-only scope. The same entry warns about a tool that reads one user’s files through an account that can read everyone’s, and the agentic list, under ASI03, about an agent that passes its full access to a helper “often for convenience”. Read-only credentials stopped every harmful call in the lab, and broke three of the four jobs.

A sandbox holds after a yes. In the coding agent, asking first only turned the upload into a question; the sandbox stopped it whatever the answer. That is the point the harness lesson made, and it scales: Anthropic reports that running Claude Code’s commands in a sandbox “safely reduces permission prompts by 84%” in its internal use, and stresses that a sandbox needs both file system and network isolation (Anthropic, October 2025). Its limits are those of any boundary: whatever is on the allowed list is still reachable, and ATLAS lists escaping a sandbox to the host machine (AML.T0105) as Realized.

Approvals depend on attention. An approval stops an action only if the person reads it and says no. Anthropic’s post names the failure: constant prompts “can lead to ‘approval fatigue’, where users might not pay close attention to what they’re approving.” Older evidence from browsers is related. In a study of over 25 million browser warnings, people clicked through 70.2% of Chrome’s certificate warnings, and between 9% and 23% of malware and phishing warnings (Akhawe and Felt, USENIX Security 2013). Warnings worked often, and failed often, and their design changed the rate; for the high one, the authors suggest frequent false alarms, and possibly warning fatigue, as causes. OWASP’s ASI09 adds an agent-specific twist: people tend to trust an agent’s confident, authoritative-sounding rationale, and an agent can offer a convincing one for a harmful action. In an injection, that rationale can come straight from the attacker’s text. OWASP’s 2026 prompt injection entry asks for confirmation of privileged, irreversible, or externally visible actions, “surfacing the exact rendered action rather than a summary to the reviewer”, and warns that “approval fatigue degrades reviewer judgment at volume.”

The lethal trifecta

In June 2025, Simon Willison named a combination of three capabilities he called the lethal trifectaLethal trifectaSimon Willison's name for an agent that can read private data and untrusted content and can send data out, so planted text can make it leak. Meta's Rule of Two builds on it: hold at most two, or have a person supervise.Open in glossary: “Access to your private data”, “Exposure to untrusted content”, and “The ability to externally communicate in a way that could be used to steal your data” (Willison, 2025). An agent with all three can be steered by the untrusted content into sending the private data out. For people who combine tools themselves, his advice is blunt: “The only way to stay safe there is to avoid that lethal trifecta combination entirely.” He is skeptical of guardrail products claiming to catch most attacks, since “in web application security 95% is very much a failing grade.”

Meta’s security team turned this into a design rule in October 2025, the Agents Rule of Two: until prompt injection can be reliably detected, an agent should have no more than two of untrusted inputs, access to sensitive systems or private data, and the ability to change state or communicate externally, within a session (Meta, 2025). If it needs all three, it should not run on its own, and at minimum needs supervision such as human approval. Meta lists the rule’s limits itself: it does not cover agent mistakes or every kind of harm, and a design that follows it can still fail if a user blindly confirms a warning.

The research assistant in the lab is that case. Its job needs all three, so every enforced control that stopped the leak also stopped the real answer, and what remained was a person reading two prompts. Some research tries to do better by design. CaMeL (Debenedetti and colleagues, 2025) has a trusted part of the system plan the steps from the user’s request alone, so data read later cannot change which steps the agent takes, and checks security policies on where data may flow whenever a tool is called. On the AgentDojo benchmark it completed 77% of tasks with what the authors call provable security, against 84% for an undefended system: a real cost in usefulness, for a real guarantee within its scope.

Standards for the checks

The controls above have to sit somewhere the model cannot talk its way past: in the harness, or in a separate service the harness asks before each action. OWASP calls this complete mediation: “Implement authorization in logic rather than relying on an LLM to decide if an action is allowed or not.”

One attempt to standardize that layer is the Agent Control Standard, created by Michael Bargury and Ory Segal and donated to the OWASP GenAI Security Project in September 2026. It specifies how an agent sends each intended action, such as a tool call, a memory write, or a retrieval, to a separate “Guardian” that answers allow, deny, modify, ask a person, or defer, with an audit trail. It is early: the current release is 0.1.3, its documentation says conformance claims are self-declared and unverified in version 0.1.0, and its own reference implementation fails open by default (if the Guardian cannot reach a decision, the action goes ahead) and does not yet authenticate its connection. A standard can make checks consistent and auditable across tools; it does not decide which policy is right for a given agent.

When one agent is not the end

Agents increasingly call other agents and tools that call more tools. OWASP’s ASI08: Cascading Failures covers a single fault, whether a mistake, a planted instruction, or poisoned memory, that “propagates across autonomous agents, compounding into system-wide harm.” The controls that limit it are the same ones at every hop: each agent gets only what its task needs, the user’s authority travels with the request rather than each agent’s own broader rights, and limits on rate and spending act as circuit breakers when something starts repeating.