Securing AI
What changes when someone attacks an AI system on purpose: instructions hidden in the data a model reads, poisoned documents and memories, agents given more power than they need, tampered training data and model files, and inputs built to fool a classifier. Each lesson maps the threat to the OWASP GenAI Security Project's lists, with NIST and MITRE ATLAS alongside, and shows what each defense catches and what it misses.
- Level
- Advanced
- Length
- 6 lessons, 2 hr 31 min
- Assumes
- The transformers track; the frontiers lessons on tools and agents help.
- Builds on
- Frontiers
- 1Attacks on AI systemsWhat changes when someone attacks an AI application on purpose, how that differs from safety and alignment, and a map of the threats from OWASP's two Top 10 lists, MITRE ATLAS, and NIST.25 min
- 2Prompt injectionA model reads instructions and data as one text, so words planted in an email or a web page can steer it. Attack a real small model in your browser and test which defenses hold.28 min
- 3Poisoned retrieval and memoryA model that reads a knowledge base or keeps memories trusts whatever lands there. Plant a post in a real retrieval pipeline, watch what it reaches, and test what each defense catches.22 min
- 4Agents with too much powerAn agent can be steered by what it reads, so the damage depends on what it is allowed to do. Change an agent's tools, credentials, sandbox, and approvals, and see what each stops and costs.22 min
- 5Poisoned data and the supply chainA model is built from data, weights, and code other people made. Poison a few training digits so a hidden trigger flips the answer, then test what inspection, clean tests, and fine-tuning catch.28 min
- 6Adversarial examplesInputs changed slightly, in exactly the wrong direction, so a classifier gets them wrong. Attack the handwriting network with its own gradients, compare random noise, and test adversarial training.26 min
Labs in this track
- Where attacks enter an AI appPick a part of an AI application and see which OWASP, MITRE ATLAS, and NIST entries apply there, or follow a documented attack from part to part.
- Instructions hidden in an emailAttack a toy email assistant running SmolLM2-135M-Instruct with lines hidden in emails, and test what prompt wording, allowlists, confirmation, and privilege separation catch.
- Poison a knowledge baseAdd one crafted post to a help bot's knowledge base, see which questions it reaches and what it puts in the prompt, and test what each defense catches.
- Agents with too much powerFour scripted agents are steered toward a harmful call. Change their tools, credentials, sandbox, and approvals, and see what stops each one and what it costs.
- Poison the training dataHide a trigger in a few training digits, train a small network in your browser, and see which checks notice the backdoor.
- Fool the handwriting networkAttack the handwriting network with its own gradients, compare random noise of the same size, and test an adversarially trained copy.