Pictures, one token at a time

In Diffusion on images a picture came out of noise everywhere at once, blurry first and sharp at the end. A chatbot works differently: it writes one token, then the next, and never goes back. This lesson runs a text-to-image modelText-to-image modelA generative model that makes a picture from a written description, called a prompt. Many, such as Stable Diffusion, are diffusion models guided by the text; others build the picture piece by piece, the way a chatbot writes.Taught in How AI sees, hears, and makes picturesOpen in glossary that makes pictures the chatbot’s way. Janus-Pro-1B, from DeepSeek, writes a picture as 576 tokens, left to right and top to bottom, with the same kind of transformer that writes text.

A vocabulary for pictures

A language model chooses each next token from a fixed vocabulary, as you saw in Predicting the next token. To write pictures the same way, you first need a vocabulary of picture pieces.

That is the job of an image tokenizer. It is an autoencoder with a twist. Its encoder shrinks a 384 by 384 picture to a 24 by 24 grid of vectors, one for each 16 by 16 square of pixels, each vector 8 numbers long. Then each vector is replaced by the nearest of 16,384 vectors in a learned list, the codebook. This step is called vector quantizationVector quantizationReplacing each vector by the nearest entry in a learned list (a codebook), so it can be stored or predicted as one whole number. Image tokenizers use it to turn each small square of a picture into one code.Open in glossary. What is left is 576 whole numbers from 0 to 16,383: the picture’s image tokensImage tokenA whole number standing for a small square of a picture: an index into an image tokenizer's codebook. A model can write image tokens one at a time, as a language model writes words, and a decoder turns them into pixels.Open in glossary, or codes. The decoder turns a grid of codes back into pixels.

Janus-Pro uses the tokenizer from LlamaGen (Sun et al., 2024), trained to rebuild pictures before Janus existed and kept fixed while Janus learned (Wu et al., 2024). It is lossy on purpose. A 384 by 384 color picture has 442,368 pixel values; its 576 codes, each one of $16{,}384 = 2^{14}$ choices, hold at most $576 \times 14 = 8{,}064$ bits, about 1 kilobyte. Whatever the codes cannot say, the decoder has to make up.

Writing the grid

Your prompt goes into the model in the chat format Janus was trained with, followed by a special token that marks the start of an image. For “A red apple on a wooden table” that is 15 tokens. Then the model starts writing codes. Each step:

1. The autoregressiveAutoregressive generationProducing a sequence one element at a time, where each new element is predicted from all the ones before it and then appended to the input.Taught in Predicting the next tokenOpen in glossary language model, 24 transformer layers, reads the prompt and every code written so far.
2. A separate image head turns its output into 16,384 scores, one per possible code.
3. The scores become probabilities, and one code is drawn at random from them.
4. That code is looked up, mapped by a small network into the model’s input space, and appended. Repeat.

Code number $i$ (counting from 0) goes to row $\lfloor i / 24 \rfloor$, column $i \bmod 24$, so after 576 steps the grid is full and the tokenizer’s decoder makes the picture. Like a chatbot, the model keeps the keys and values of everything it has read, so each step only computes the newest position (Generating fast: the KV cache).

Janus-Pro is a “unified” model: the same transformer also reads pictures and answers questions about them. For that it uses a different encoder, SigLIP, which turns a photo into features rich in meaning rather than into codes good for rebuilding pixels. Keeping the two encoders separate is the main idea of the Janus papers (Chen et al., 2025). This lab downloads only the parts that make pictures.

import numpy as np

# 1. A vector-quantized (VQ) tokenizer turns each 16 x 16 square of a picture
#    into one code: the number of the nearest entry in a learned codebook.
#    Janus-Pro's codebook has 16,384 entries of 8 numbers; this one has 4 of 2.
codebook = np.array([[0.0, 0.0], [1.0, 0.0], [0.0, 1.0], [1.0, 1.0]])
squares = np.array([[0.9, 0.2], [0.1, 0.8], [0.6, 0.7]])  # the encoder's output
dist = ((squares[:, None, :] - codebook[None, :, :]) ** 2).sum(axis=-1)
print("codes:", dist.argmin(axis=1))

# 2. The model writes codes in reading order: code i lands in row i // 24,
#    column i % 24 of the 24 x 24 grid, so 576 codes make a 384 x 384 picture.
for i in (0, 23, 24, 287, 575):
    print(f"code {i:3d} -> row {i // 24:2d}, column {i % 24:2d}")

# 3. For each code, guidance mixes the logits computed with the prompt (cond)
#    and with the prompt replaced by pad tokens (uncond).
cond = np.array([1.2, 0.9, 0.3, 0.0])
uncond = np.array([0.6, 0.6, 0.5, 0.5])


def probs(s):
    logits = uncond + s * (cond - uncond)
    p = np.exp(logits - logits.max())
    return p / p.sum()


for s in (0, 1, 5):
    p = probs(s)
    print(f"guidance {s}: p = {np.round(p, 3)}, spread {-(p * np.log2(p)).sum():.2f} bits")

# 4. Sampling picks each code at random from p; a seed fixes the random
#    numbers, so the same seed picks the same codes again.
for seed in (0, 0, 1):
    rng = np.random.default_rng(seed)
    print(f"seed {seed}: codes", rng.choice(4, size=10, p=probs(1)))

codes: [1 2 3]
code   0 -> row  0, column  0
code  23 -> row  0, column 23
code  24 -> row  1, column  0
code 287 -> row 11, column 23
code 575 -> row 23, column 23
guidance 0: p = [0.262 0.262 0.238 0.238], spread 2.00 bits
guidance 1: p = [0.408 0.303 0.166 0.123], spread 1.85 bits
guidance 5: p = [0.804 0.179 0.013 0.003], spread 0.81 bits
seed 0: codes [1 0 0 0 2 3 1 2 1 3]
seed 0: codes [1 0 0 0 2 3 1 2 1 3]
seed 1: codes [1 3 0 3 0 1 2 1 1 0]

What you just saw

The grid fills in reading order. The squares light up left to right, row by row. Their colors come from running the decoder on the codes made so far, every four rows; the sharp picture appears only when all 576 codes exist. Drag the slider back to code 288 and exactly the top 12 rows are decided. The model wrote them without knowing what the bottom half would hold, and it will never change them.

The seed decides which code is drawn, not what is likely. At every square the model gives each of the 16,384 codes a probability, and a random number picks one. A seedRandom seedA number that sets where a computer's random number generator starts. The same seed gives the same "random" numbers every time, so a result that uses randomness can be repeated exactly.Taught in How AI sees, hears, and makes picturesOpen in glossary fixes that sequence of random numbers. In the recordings, a different seed picked a different code at the first or second square for every example, and from there every later square was drawn from a different past: same prompt, different picture. The same seed, prompt, and settings gave the same 576 codes again on the device we recorded with. The probabilities themselves depend on how the hardware rounds its arithmetic, though. Run on a CPU at full precision instead of the browser’s 4-bit model on a GPU, the same seed picked a different first code and drew a different apple.

One word changes almost everything. The red apple and the green apple with seed 0 share their first code and differ from the second on. Nothing in the method keeps the rest of the picture the same when the prompt changes.

Guidance narrows each choice. The lab shows the spread of each code’s probabilities as entropy: 14 bits if all 16,384 codes were equally likely. In the recordings the average was about 10.5 bits at guidance 0, 9.0 to 9.6 at 1, 5.0 to 6.7 at the default of 5, and 4.3 to 5.1 at 8. At guidance 0 the pictures have nothing to do with the prompt: a boxy structure on pink for the apple, a dark figure on blue for the lighthouse. The site’s CLIP model (from One space for images and text) agrees: it rates those two as matching their prompts at a cosine similarity of 0.18, against 0.29 to 0.37 for every other recorded picture. Between guidance 1 and 8 its ratings barely move, and one picture per setting is too few to rank them. Stronger guidance also costs variety: the three cats at guidance 5 sit in nearly the same pose.

Guidance costs time. Guidance needs the model’s prediction with and without the prompt, so each step runs two rows instead of one. The GPU runs them side by side, but it is still slower: in the recordings the codes took about 1.7 times as long with guidance as at 0 or 1. A whole picture took between about half a minute and a minute on a 2022 laptop (Apple M2), depending on what else the machine was doing.

Guidance on logits

Classifier-free guidanceClassifier-free guidanceA sampling technique for conditional generative models, diffusion or token by token, that compares the model's prediction with and without the prompt and pushes further toward the prompt, trading variety for closer matching.Taught in Diffusion on imagesOpen in glossary works here as it did for the digit diffusion model, applied to different numbers. For every code the model runs twice: once with your prompt, giving logits $l_c$, and once with the prompt replaced by pad tokens, giving $l_u$. The logits it samples from are

with $s = 5$ by default (Wu et al., 2024, section 3.4). The pad version is meaningful because during training 10% of the prompts were replaced by pad tokens, so the model also learned to draw with no prompt. In the diffusion lesson the same formula combined two noise estimates; here it combines two sets of 16,384 scores, one set per code. Then a softmax at temperature 1, as in DeepSeek’s example code, turns the guided logits into probabilities.

Diffusion and token by token, side by side

Writing pictures as tokens is not new. OpenAI’s first DALL-E (2021) did it with a 12-billion-parameter transformer, writing a 32 by 32 grid of codes from a vocabulary of 8,192 after up to 256 text tokens (Ramesh et al., 2021). OpenAI says the image generation built into GPT-4o “is an autoregressive model”, unlike DALL·E (meaning its later DALL·E models, not the first), “which operates as a diffusion model” (OpenAI, March 2025); it has not published how the model works in detail.

Where it goes wrong

Counts, positions, and words. GenEval checks generated pictures with an object detector. On it, Janus-Pro-1B got single objects right 98% of the time, two objects 82%, colors 89%, but positions only 65%, colors bound to the right object 56%, and counts 51%, for an overall score of 0.73 (the 7-billion-parameter version scored 0.80; Chen et al., 2025, Table 4). The recorded ducks show it: asked for five in a row, we count about six, four, and six, and not in a row. One of the three green apples is two apples. Words are harder still: one of the three recorded signs has OPEN on it; another reads OAE N.

Fine detail. Every 16 by 16 square has to be one of 16,384 codes, and the picture is only 384 by 384. The Janus-Pro paper names both as the reason its pictures “still lack fine details”, with small faces as the example.

No going back. A code that starts a mistake stays, and every later code is conditioned on it. The Janus paper lists combining left-to-right generation with steps that look at the whole grid as one way “to reduce accumulated errors” (Wu et al., 2024, section 3.5).

The browser’s copy is smaller than the paper’s. The paper’s scores are for the full model. To fit a browser, this lab runs a 4-bit version of the language model (698 MB instead of about 5 GB at 32 bits). We compared the two on the same prompts and codes. On a CPU, the 4-bit model’s probabilities for each code differ from the full model’s by 0.84 bits on average (KL divergence), and its single most likely code is the same only 45% of the time. Run as this lab runs it, on a GPU in the browser, the difference is a little larger: 1.30 bits over the first 192 codes of the same three pictures, against 1.14 on the CPU. Its pictures look like the full model’s in kind, but they are its own.

Token by token in 2026

Janus-Pro writes codes strictly one at a time, in reading order, and never revises them, the third limit above. Other published designs change those choices, some from before Janus-Pro and some from after. As of October 2026, three directions stand out among models whose papers and weights are public.

Coarse to fine instead of reading order. VAR (Tian et al., 2024) keeps the autoregressive chain rule but changes what one step is: it predicts a whole grid of codes at a time, starting from a single code for the whole picture, then larger ones for finer detail, each conditioned on the coarser ones. The authors call this “next-scale prediction” and report that it beat a diffusion transformer on ImageNet in quality and speed.

Many codes at once. Masked models such as MaskGIT (Chang et al., 2022) fill in a grid of codes over a few rounds, keeping the most confident guesses each round. Emu3.5 (Cui et al., October 2025) shows the two can be combined: trained with plain next-token prediction on more than 10 trillion tokens of text and pictures, it is then adapted to predict codes in parallel, which its authors report makes each picture about 20 times faster “without sacrificing performance”.

Diffusion for the pixels, tokens for the words. Other models keep next-token prediction for text and hand the picture to diffusion inside the same transformer, the Transfusion design described in Where multimodal models are heading. ByteDance’s BAGEL (Deng et al., 2025) generates with rectified flow on an autoencoder’s latents and reads pictures with SigLIP 2, much as Janus-Pro reads with SigLIP. Tencent’s HunyuanImage 3.0 (September 2025) does the same inside a mixture of experts with more than 80 billion parameters, 13 billion of them used per token.

Closed systems say less. OpenAI’s system card for ChatGPT Images 2.0 (April 2026) describes a “thinking mode” that adds reasoning and tool use to image generation, but not whether the pictures are written as tokens or denoised. Google’s Gemini 3 Pro model card lists Gemini 3 Pro Image as a model based on Gemini 3 Pro, without saying how it makes pictures. So from outside it is not known which of these designs the image generators in ChatGPT and Gemini use today.

Who made this picture?

Save a picture from this lab and nothing in the file says a model made it. The lab adds no watermarkWatermarkA signal a model hides in what it generates, such as a statistical preference for certain words or a pattern in pixels, that a detector with the right key can test for later.Taught in Safety in AI productsOpen in glossary and no content credentialContent credentialsA signed record attached to an image, video, audio, or document file saying who made it, with what tools, and how it was edited (the C2PA standard). It shows tampering, not truth, and can be removed.Taught in Safety in AI productsOpen in glossary; Safety in AI products explains what those can and cannot prove, and the laws that are starting to require them. Larger models make pictures that pass for photos, which is why provenance matters.

What the lab does instead is check every picture on your device with a small safety classifierSafety classifierA separate model that checks what goes into or comes out of a language model against a policy, and blocks or flags what breaks it. Like any classifier, it makes errors both ways.Taught in Safety in AI productsOpen in glossary, image-safety-classifier-xs (3.5 million parameters, MIT license), which sorts pictures into sexual content, gore, and everything else. If the first two together are rated more likely than not, the picture is blurred and the reason shown. There is no list of forbidden words; a prompt is never refused, only the picture judged. Its model card reports 97.76% accuracy on its author’s own data. We checked it on safe pictures:

• On 1,000 everyday photos from the Flickr30k test set it blurred 14 (1.4%). Five show people in swimwear or without a shirt and seven show children in everyday clothes; the others are a man shaving and a soldier inspecting an old gun (rated as gore).
• Of 20 pictures from Janus-Pro for safe prompts chosen to be hard (beaches, a surgeon, raw meat, boxing), it blurred 2: a woman in a bikini, and a swimmer whose generated body came out distorted and without a visible swimsuit. It blurred none of the 26 recorded pictures.

Its mistakes are not random: they cluster on bare skin and on children. And it was trained to look for nothing else, so a fake photo of a real event or a hateful symbol passes it. The model’s license adds its own rules. Janus-Pro’s weights come with the DeepSeek Model License, whose use restrictions apply to anything you make with it, such as no content that harms minors, harasses or defames people, or spreads false information to harm others.