The forward process of a diffusion model, on a real 28 × 28 digit from the MNIST test set. The network below is the small one this site trained for the lesson; it runs in your browser from a 1 MB weight file. At each step it sees only the noisy image and the step number (and the class, if you tell it), and guesses which part of the image is noise.
Noising a digit
Add noise to a real handwritten digit step by step, and see what the trained network guesses at each step.
Loading the model (1.3 MB)…
Challenges
0 of 2 complete
- Find the step where the digit and the noise are equally strong. (not yet)
- Past step 500, find a step where the recognizer reads the clean estimate correctly when the model is told the digit, but not when the digit is hidden. (not yet)
Try this
- Drag Step t from 0 to 1000 and watch the chart: the digit fades as a curve and the noise grows as the other.
- Stop at step 100, then step 500. Compare the network’s noise estimate with the true noise above it.
- Go to step 700, then switch Tell the model which digit it is off and on.
- Press New noise at step 500: a different noise sample gives a different clean estimate.
The forward process and the clean estimate in NumPyPython with NumPy
import numpy as np
# DDPM's linear schedule: beta rises from 0.0001 to 0.02 over 1000 steps.
betas = np.linspace(1e-4, 0.02, 1000)
abar = np.cumprod(1 - betas) # share of the signal's variance left after step t
def noisy(x0, eps, t): # the forward process, t from 1 to 1000
a = abar[t - 1]
return np.sqrt(a) * x0 + np.sqrt(1 - a) * eps
for t in (1, 260, 500, 1000):
a = abar[t - 1]
print(f"t = {t:4d}: signal kept {np.sqrt(a):.3f}, noise {np.sqrt(1 - a):.3f}")
# A 4-pixel "image" on the model's [-1, 1] scale, and the noise mixed into it.
x0 = np.array([-1.0, -1.0, 0.6, 1.0])
eps = np.array([0.5, -1.2, 0.3, 2.0])
xt = noisy(x0, eps, 500)
print("x_500:", np.round(xt, 3))
# If you knew the noise exactly, you could undo the mixing: this is the
# "clean estimate" the lab shows, computed from the network's noise guess.
a = abar[499]
print("clean estimate:", np.round((xt - np.sqrt(1 - a) * eps) / np.sqrt(a), 3))t = 1: signal kept 1.000, noise 0.010
t = 260: signal kept 0.705, noise 0.709
t = 500: signal kept 0.280, noise 0.960
t = 1000: signal kept 0.006, noise 1.000
x_500: [ 0.2 -1.432 0.456 2.2 ]
clean estimate: [-1. -1. 0.6 1. ]The network never sees the clean digit or the true noise. Its clean estimate is computed from its noise estimate with the same formula the code uses, so a small error in the noise estimate becomes a large error in the clean estimate when little signal is left.