Steering a model

Build a direction in the model's residual stream from two sets of prompts, add it while the model writes, and see what changes.

Recorded from SmolLM2-135M

Loading the model's recorded outputs for the default sets and start of the text, at every layer and strength.

About the download

Steer a real language model by adding a direction to its residual stream while it writes. This uses SmolLM2-135M, a 135-million-parameter base language model from Hugging Face (2024), run from its original weights by this site's own engine.

The first time, your browser downloads about 272 MB of model files from Hugging Face. They are cached for later visits. Everything runs on your device; nothing you type is sent anywhere.

Manage downloads

Loading the recorded results

Challenges

0 of 3 complete

Find a layer and strength where steering changes what the model writes. (not yet)
Change the text with a positive strength, then with a negative one. (not yet)
Try steering at an early layer (below 6) and at a late one (above 20). (not yet)

Try this

Press Generate with the defaults (after layer 4, strength +2), then again at -2. The same direction pushes the text both ways.
Raise the strength to +8. The text stops reading as language.
Try layer 20. On the default prompt, +2 still reads warmer, but -2 breaks the text.
Replace both sets with your own, for example sentences about the sea against sentences about the desert.

A steering vector in a few linesPython with NumPy

import numpy as np

# Residual streams (length 4) for prompts on each side, averaged per side.
happy = np.array([[0.9, 0.1, 0.3, 0.2],
                  [0.8, 0.0, 0.4, 0.1]])
angry = np.array([[0.1, 0.9, 0.3, 0.2],
                  [0.0, 0.8, 0.5, 0.1]])
vector = happy.mean(axis=0) - angry.mean(axis=0)
print("steering vector:", np.round(vector, 2))

# Add it, scaled, to a new prompt's residual stream and read the output.
W_U = np.array([[1.0, -1.0, 0.0, 0.0],   # " kind"
                [-1.0, 1.0, 0.0, 0.0]])  # " rude"
h = np.array([0.4, 0.5, 0.3, 0.2])
for strength in (0, 1, -1):
    logits = W_U @ (h + strength * vector)
    p = np.exp(logits - logits.max())
    p /= p.sum()
    print(f"strength {strength:+d}: P(kind) = {p[0]:.2f}")

steering vector: [ 0.8  -0.8  -0.05  0.  ]
strength +0: P(kind) = 0.45
strength +1: P(kind) = 0.95
strength -1: P(kind) = 0.03