Model microscope

Type a prompt and look inside a real language model: what it would say at every layer, where each attention head looks, and what happens when you switch parts off.

Recorded from SmolLM2-135M

Loading the model's recorded outputs for four prompts: every layer and head, tracking the tokens near the top at the last position, every single head switched off, and patching at every layer.

About the download

See inside a real language model: its prediction at every layer, all 270 attention heads, and what happens when you switch parts off. This uses SmolLM2-135M, a 135-million-parameter base language model from Hugging Face (2024), run from its original weights by this site's own engine.

The first time, your browser downloads about 272 MB of model files from Hugging Face. They are cached for later visits. Everything runs on your device; nothing you type is sent anywhere.

Manage downloads

Loading the recorded results

Challenges

0 of 3 complete

With the model's answer tracked, select the first layer where it becomes the top guess at the last token. (not yet)
Switch off one single head that cuts the tracked answer's probability by at least a third. (not yet)
Patch one layer from the second prompt so the model's top answer changes. (not yet)

Try this

In Layer by layer, the model’s answer, " Paris", is tracked from the start. Color the cells by its probability and read the rank chart: it becomes the top guess only from layer 28.
Switch to Attention. The selected head, layer 17, head 1, mostly looks at the token just before the one doing the looking.
In Intervene, sweep every layer with patching at the last token, then again where the prompts differ. The two curves trade places at the same layer, 24.
Switch off layer 2, head 1 and run. " Paris" falls from 88% to 16%.

The logit lens in a few linesPython with NumPy

import numpy as np

# A toy model: 4 vocabulary tokens, residual vectors of length 3.
vocab = ["Paris", "Rome", "the", ","]
W_U = np.array([[2.0, 0.0, 1.0],    # unembedding: one row per token
                [0.0, 2.0, 1.0],
                [0.5, 0.5, 0.0],
                [0.0, 0.0, 0.5]])
g = np.array([1.0, 1.0, 1.0])       # final RMSNorm weight

def rms_norm(x, eps=1e-5):
    return x / np.sqrt(np.mean(x ** 2) + eps) * g

def lens(h):
    logits = W_U @ rms_norm(h)
    p = np.exp(logits - logits.max())
    return p / p.sum()

# The residual stream at three depths for the last token.
stream = {"layer 0": np.array([0.9, 0.9, -1.4]),
          "layer 15": np.array([0.2, 1.0, 0.2]),
          "output": np.array([1.5, 0.1, 0.3])}

for name, h in stream.items():
    p = lens(h)
    best = int(p.argmax())
    print(f"{name}: {vocab[best]} {p[best]:.2f}")

layer 0: the 0.40
layer 15: Rome 0.86
output: Paris 0.88