Where attacks enter an AI app
Pick a part of a typical AI application to see which entries of OWASP's two lists, MITRE ATLAS, and NIST apply there, or follow a documented attack through the parts.
Tools. Actions the app runs when the model asks: sending email, running code, calling other services.
OWASP Top 10 for LLM Applications 2026
- LLM03 Excessive Agency. The app lets the model do more than the task needs, so a mistake or an injection becomes a damaging action. Lesson: Agents with too much power
- LLM06 Unbounded Consumption. Nothing caps the work a request can cause, so attackers run up costs, deny service, or copy the model by querying it. Lesson: Inside an agent harness, Compute and energy
OWASP Top 10 for Agentic Applications 2026
- ASI02 Tool Misuse and Exploitation. The agent uses a tool it is allowed to use in a harmful way: deleting, over-calling, or sending data out. Lesson: Agents with too much power
- ASI03 Identity and Privilege Abuse. The agent acts with borrowed or inherited credentials, so it can do what neither it nor the attacker should. Lesson: Agents with too much power
- ASI05 Unexpected Code Execution (RCE). Text turns into code that runs on a machine: generated, injected, or unpacked from a file. Lesson: Agents with too much power, Inside an agent harness
MITRE ATLAS techniques, with ATLAS's evidence level
- AML.T0053 AI Agent Tool Invocation Demonstrated
- AML.T0086 Exfiltration via AI Agent Tool Invocation Realized
- AML.T0101 Data Destruction via AI Agent Tool Invocation Realized
NIST AI 100-2 E2025 attack names
- NISTAML.039 Compromising connected resources
Documented attacks through this part
- (2025, ATLAS incident)
- (2025, ATLAS incident)
OWASP LLM entries2
OWASP agentic entries3
ATLAS techniques shown3
Challenges
0 of 3 complete
- Find a documented attack that ATLAS records as a real incident, not a researchers' exercise. (not yet)
- Follow an attack that passes through at least three parts of the app. (not yet)
- Find a part where ATLAS rates a technique Feasible: shown in research, with no recorded use by an attacker. (not yet)
Try this
- Pick each part in turn and count the OWASP entries. Which part collects the most, and which entries appear on more than one part?
- Compare the ATLAS evidence levels on Input and Supply chain. Which risks has ATLAS seen real attackers use?
- Switch to Follow an attack and sort the eight into incidents and exercises.
Which attacks a defense at one part would stopPython with the standard library
# Documented attacks and the parts of an app each passed through
# (MITRE ATLAS case studies; "incident" means a real attacker, "exercise" researchers).
ATTACKS = {
"EchoLeak (exercise)": ["input", "retrieval", "output"],
"Slack AI (exercise)": ["input", "retrieval", "output"],
"ChatGPT memories (exercise)": ["input", "memory"],
"Summarize-with-AI links (incident)": ["input", "memory", "output"],
"postmark-mcp package (incident)": ["supply", "tools"],
"Model files that run code (incident)": ["supply", "model"],
"Amazon Q extension (incident)": ["supply", "tools"],
"Morris II worm (exercise)": ["input", "retrieval", "output", "agents"],
}
def stopped(defended):
# Suppose a defense at a part always holds: an attack is stopped
# if it passes through at least one defended part.
return sum(bool(set(path) & defended) for path in ATTACKS.values())
for defended in [{"input"}, {"output"}, {"supply"}, {"input", "supply"}]:
print(f"defend {sorted(defended)}: stops {stopped(defended)} of {len(ATTACKS)}")
# Real layers are not perfect. If each of three layers stops 70% of attempts
# and they fail independently, an attempt gets through all three with:
p_through = 0.3 ** 3
print(f"three 70% layers, independent: {p_through:.1%} get through")
# Layers that share a weakness (all fooled by the same wording) do much worse.defend ['input']: stops 5 of 8
defend ['output']: stops 4 of 8
defend ['supply']: stops 3 of 8
defend ['input', 'supply']: stops 8 of 8
three 70% layers, independent: 2.7% get through