Generating a digit

A U-Net with about half a million parameters, trained by this site on the 60,000 MNIST training digits, generates a 28 × 28 digit of the class you choose. Every step runs the real network in your browser (a 1 MB weight file, no server). At each step you can see the image it was given, its estimate of the clean image, and its estimate of the noise.

Generating a digit

A trained network turns noise into a digit of the class you ask for, one denoising step at a time.

Starting noise

Its clean estimate x̂₀

Its noise estimate

Class to generate

Guidance scale3.0

Sampler

DDIMDDPM

Steps20

Starting noise#1

Network calls40

Recognizer readsn/a

Time (estimate)n/a

Challenges

0 of 3 complete

Generate a digit the recognizer reads as the class you asked for (90% or more), using DDIM with 10 steps or fewer. (not yet)
Keep one starting noise and get 3 different digits from it, each read correctly. (not yet)
Find a starting noise and class where guidance 1 gives a digit the recognizer misreads, then raise the guidance until it reads correctly. (not yet)

Try this

Press Generate and drag Show step back to the start. The clean estimate is a blur at first, then a rough digit, then the final one.
Switch to DDIM with 10 steps and generate again from the same starting noise, then with 100 steps. The digits are nearly the same.
Switch to DDPM with 1000 steps, then press New DDPM randomness and run again. Same starting noise, a differently drawn digit each time.
Keep the starting noise and change only the class.
Switch back to DDIM with 20 steps, press New starting noise once (noise #2), set guidance to 1, and generate a 2. The recognizer misreads it; raise the guidance and generate it again.
Set guidance to 0, then 6, with everything else fixed.

One sampling step and guidance in NumPyPython with NumPy

import numpy as np

betas = np.linspace(1e-4, 0.02, 1000)
abar = np.cumprod(1 - betas)


def step(x, eps_hat, t, t_prev, eta, z):
    """One reverse step from t to t_prev (Song et al. 2020, eq. 12).
    t_prev = 0 means the clean image. eta = 0 is DDIM, eta = 1 is DDPM."""
    a = abar[t - 1]
    a_prev = abar[t_prev - 1] if t_prev > 0 else 1.0
    x0_hat = np.clip((x - np.sqrt(1 - a) * eps_hat) / np.sqrt(a), -1, 1)
    eps_hat = (x - np.sqrt(a) * x0_hat) / np.sqrt(1 - a)  # agree with the clipped x0
    sigma = eta * np.sqrt((1 - a_prev) / (1 - a) * (1 - a / a_prev))
    return np.sqrt(a_prev) * x0_hat + np.sqrt(1 - a_prev - sigma**2) * eps_hat + sigma * z


# Classifier-free guidance: the network runs twice and the answers are combined.
eps_cond = np.array([0.4, -0.9, 0.1, 1.6])  # told "this is a 3"
eps_uncond = np.array([0.6, -0.7, 0.2, 1.2])  # told nothing
for scale in (0, 1, 3):
    print(f"guidance {scale}:", np.round(eps_uncond + scale * (eps_cond - eps_uncond), 2))

x = np.array([0.2, -1.4, 0.5, 2.2])  # a 4-pixel image at step t = 500
eps_hat = eps_uncond + 3 * (eps_cond - eps_uncond)
z = np.array([0.3, 0.3, -0.8, 0.1])  # fresh noise, used only by DDPM
print("DDIM, 500 to 400:", np.round(step(x, eps_hat, 500, 400, 0.0, z), 3))
print("DDPM, 500 to 499:", np.round(step(x, eps_hat, 500, 499, 1.0, z), 3))

guidance 0: [ 0.6 -0.7  0.2  1.2]
guidance 1: [ 0.4 -0.9  0.1  1.6]
guidance 3: [ 0.  -1.3 -0.1  2.4]
DDIM, 500 to 400: [ 0.315 -1.406  0.647  1.99 ]
DDPM, 500 to 499: [ 0.231 -1.363  0.42   2.196]

The recognizer that reads each result is the small network from the Reading handwriting lesson (97.8% accurate on real test digits), so a misreading can be the recognizer’s mistake as well as the generator’s.