Inside the model
Open up a real language model running in your browser. Read its predictions layer by layer, switch off attention heads to see what they do, watch an induction circuit form during training, and steer the model's output. Then see why understanding and aligning models is hard.
- Level
- Advanced
- Length
- 6 lessons, 2 hr 45 min
- Assumes
- The transformers track.
- Builds on
- Transformers and LLMs
- 1Reading a model layer by layerWatch a real language model build its answer across 30 layers. Read its guess at every depth with the logit lens, and see where each attention head looks.25 min
- 2How circuits form: induction headsTrain a tiny transformer in your browser and watch it suddenly learn to copy, as two attention heads in two layers lock together into an induction circuit.25 min
- 3Superposition: more features than neuronsTrain a tiny model that squeezes five features into two dimensions, and see why packing features this way makes neurons hard to read.25 min
- 4Switching parts off and steeringChange a real model while it runs. Switch attention heads off, copy activations from one prompt into another to see where a fact travels, and add a direction that changes what the model writes.30 min
- 5Alignment: getting the goal rightA model can learn the wrong goal even from a correct reward. Watch an agent ace training and then walk past the coin, and see how evaluations and red teaming look for such failures.25 min
- 6How alignment is attemptedThe methods developers use to make models do what people intend, the evidence that each one works, the limits their own authors report, and where experts disagree about long-term risk.35 min
Labs in this track
- Model microscopeLook inside a real language model in your browser. Read its guess at every layer, see all 270 attention heads, and switch parts off.
- Watch an induction circuit formTrain a two-layer transformer in your browser and watch copying appear suddenly, then switch heads off to find what the circuit depends on.
- Five features, two dimensionsTrain a toy ReLU model in your browser and watch sparse features share directions, as antipodal pairs and then a pentagon.
- Steering a modelBuild a direction from two sets of prompts, add it to a real model's residual stream while it writes, and compare the text with and without it.
- Coin or door?Train an agent whose coin almost always sits on the door, then move the coin and watch it keep its skills but head for the wrong goal.
- Teach a reward modelLabel pairs of replies, fit a Bradley-Terry reward model to your choices, and watch a policy that maximizes it flatter users who are wrong.