A chat, as the model reads it

Every turn, the app turns the whole conversation into one text with the chat template and sends it again; the model only ever writes the next tokens.

Recorded from SmolLM2-135M-Instruct

Loading the model's recorded outputs for a five-message conversation with each system prompt and window.

About the download

Chat with a small instruction-tuned model and see exactly what it reads each turn. This uses SmolLM2-135M-Instruct, the chat version of SmolLM2-135M from Hugging Face (2024): the same 135 million parameters, fine-tuned on about 460,000 example conversations and then preference-tuned, run from its original weights by this site's own engine.

The first time, your browser downloads about 272 MB of model files from Hugging Face. They are cached for later visits. Everything runs on your device; nothing you type is sent anywhere.

Manage downloads

Loading the recorded results

Challenges

0 of 3 complete

Ask the model who it is with no system prompt of your own, then again with one. (not yet)
Get the model to tell you what year it is. (not yet)
Keep chatting until the app has to drop your first message to make room. (not yet)

The model always takes its most likely next token, so sending the same conversation again gives the same reply (another browser can, rarely, choose differently where two tokens are nearly tied). A reply ends when the model writes its end-of-turn token or reaches 96 tokens, and the app keeps that much of the window free for it. SmolLM2 was trained to read 8,192 tokens; the windows here are smaller so you can watch one fill.

Try this

Send Who are you? with no system prompt, then pick Pirate and send it again. The name the model gives comes from the system prompt.
Pick Today’s date and send messages until What year is it?. Then pick None and do the same.
With a 256-token window, keep sending messages and watch the oldest turns drop out of what the model reads.

The chat template, and dropping old turns to fitPython with the standard library

# SmolLM2-135M-Instruct's chat template, written out by hand.
DEFAULT_SYSTEM = "You are a helpful AI assistant named SmolLM, trained by Hugging Face"

def piece(role, content):
    return f"<|im_start|>{role}\n{content}<|im_end|>\n"

def render(system, turns, message):
    text = piece("system", system or DEFAULT_SYSTEM)  # inserted when none is given
    for user, assistant in turns:
        text += piece("user", user) + piece("assistant", assistant)
    return text + piece("user", message) + "<|im_start|>assistant\n"

turns = [
    ("Hi! My name is Priya.", "Hello, Priya! How can I help you today?"),
    ("Give me one tip for sleeping better.",
     "Go to bed and wake up at the same time every day, even on weekends."),
]
message = "What is my name?"

# Tokens in each piece, counted with SmolLM2's tokenizer. Special tokens split
# the text, so the pieces' counts add up to the whole prompt's.
count = {"system": 21, "header": 4, "What is my name?": 10,
         turns[0]: 13 + 18, turns[1]: 13 + 23}

window, reply_limit = 128, 32  # the app keeps room for the reply
kept = list(turns)
while True:
    used = count["system"] + sum(count[t] for t in kept) + count[message] + count["header"]
    if used + reply_limit <= window or not kept:
        break
    kept.pop(0)  # drop the oldest turn

print(f"{len(turns) - len(kept)} turn dropped; the prompt is {used} tokens:")
print(render(None, kept, message))

1 turn dropped; the prompt is 71 tokens:
<|im_start|>system
You are a helpful AI assistant named SmolLM, trained by Hugging Face<|im_end|>
<|im_start|>user
Give me one tip for sleeping better.<|im_end|>
<|im_start|>assistant
Go to bed and wake up at the same time every day, even on weekends.<|im_end|>
<|im_start|>user
What is my name?<|im_end|>
<|im_start|>assistant