A live language model

A real language model, running in your browser

SmolLM2-135M reads your text and scores every token in its vocabulary as the next one.

Recorded from SmolLM2-135M

Loading the model's recorded outputs for the texts below, one token on from each, and the tokens Sample and Take the most likely add to them.

About the download

Type any text and see what a real language model predicts next, with its actual probabilities. This uses SmolLM2-135M, a 135-million-parameter base language model from Hugging Face (2024), run from its original weights by this site's own engine.

The first time, your browser downloads about 272 MB of model files from Hugging Face. They are cached for later visits. Everything runs on your device; nothing you type is sent anywhere.

Manage downloads

Loading the recorded results

Temperature1.00

Top-k20

Top-p1 (off)

Uncertainty of the full distributionn/a

Probability held by the top 20n/a

Tokens added by the model0

Challenges

0 of 4 complete

Find a text after which the model gives one token more than 90% probability. (not yet)
Find a text where the 20 most likely tokens hold less than half of the probability. (not yet)
Make Paris the model's top next token, with more than 50% probability. (not yet)
Type a single word that the model reads as four or more tokens. (not yet)

Bar lengths are the model's own probabilities over all 49,152 tokens at temperature 1. Your settings are applied to the 20 most likely tokens, renormalized among themselves; tokens outside the top 20 are never sampled here. Sampling uses a fixed seed, so Reset replays the same choices.

Try this

Compare the uncertainty readout for Once upon a with The capital of France is.
Set temperature to 0 and generate twice, then set it to 1.5.
Ask a question, then rewrite it as a “Q: … A:” document and compare.

From logits to the numbers in the labPython with NumPy

import numpy as np

# Made-up logits for a 10-token vocabulary (the real model scores 49,152).
logits = np.array([6.0, 3.5, 3.0, 2.0, 1.5, 1.0, 0.5, 0.0, -0.5, -1.0])

def softmax(z):
    e = np.exp(z - z.max())              # subtract the max for numerical safety
    return e / e.sum()

p = softmax(logits)                      # the model's own probabilities
H = -(p * np.log2(p)).sum()              # uncertainty (entropy) in bits
print(f"top token {p[0]:.1%}, uncertainty {H:.2f} bits, like {2**H:.1f} equal choices")

# The lab receives only the most likely tokens (20; 3 here) and applies
# temperature to their logits, so their chances add up to 1 among themselves.
top = np.argsort(-logits)[:3]
print(f"held by the top 3: {p[top].sum():.1%}")
for T in (1.0, 0.5, 1.5):
    print(f"temperature {T}:", np.round(softmax(logits[top] / T), 3))

top token 85.0%, uncertainty 0.93 bits, like 1.9 equal choices
held by the top 3: 96.2%
temperature 1.0: [0.883 0.073 0.044]
temperature 0.5: [0.991 0.007 0.002]
temperature 1.5: [0.755 0.143 0.102]

This is a base model: it continues text rather than following instructions. Probabilities are the model’s own, over its full 49,152-token vocabulary; sampling here draws from the 20 most likely tokens.