Diffusion: from noise to data

Many of today’s image and video generators, including Stable Diffusion, are diffusion modelsDiffusion modelA generative model that learns to reverse a gradual noising process, so it can turn pure random noise into new data such as images, step by step.Open in glossary or close relatives of them. Given a prompt, they do not paint from left to right. They start from a canvas of pure random noise and, over a few dozen steps, remove a little noise at a time until a picture is left. This lesson shows why that works, using data simple enough that the hard part can be computed exactly.

Destroying data is easy

Start with the easy direction. Take a clean data point $x_0$ and blend it with Gaussian noise $\varepsilon \sim \mathcal{N}(0, I)$:

Here $t$ runs from 0 to 1 and $\bar\alpha_t$ (read “alpha bar”) falls smoothly from 1 to nearly 0. At $t = 0$ you have the data. At $t = 1$ you have noise that no longer remembers where it came from. Because the signal shrinks while the noise grows, a sample with unit variance keeps unit variance throughout, which is why this is called a variance-preserving process. This is the forward process of DDPM (Ho et al., 2020), written in the continuous-time form of Song et al. (2021); the demo uses their schedule, with $\beta(t)$ rising linearly from 0.1 to 20 and $\bar\alpha_t = e^{-\int_0^t \beta(s)\,ds}$.

Below, every dot is a sample from a 2D dataset made of small Gaussian blobs arranged in a shape. Set the direction to Add noise first.

import numpy as np

# The demo's schedule: beta(t) rises from 0.1 to 20 as t goes from 0 to 1.
def beta(t):
    return 0.1 + 19.9 * t

def alpha_bar(t):
    return np.exp(-(0.1 * t + 0.5 * 19.9 * t**2))

# Forward process: keep sqrt(abar) of a clean point, add sqrt(1 - abar) of noise.
x0, eps = np.array([1.5, 0.0]), np.array([0.3, -1.2])
for t in (0.1, 0.3, 1.0):
    ab = alpha_bar(t)
    xt = np.sqrt(ab) * x0 + np.sqrt(1 - ab) * eps
    print(f"t = {t}: signal kept {np.sqrt(ab):.3f}, x_t = ({xt[0]:.3f}, {xt[1]:.3f})")

# Data: two small Gaussian blobs. Noising them gives two wider, shrunken
# blobs, so the score (the gradient of log density) has an exact formula.
means, std = np.array([[-1.0, 0.0], [1.0, 0.0]]), 0.1

def score(x, t):
    ab = alpha_bar(t)
    m, v = np.sqrt(ab) * means, ab * std**2 + 1 - ab
    logits = -np.sum((x - m) ** 2, axis=1) / (2 * v)
    r = np.exp(logits - logits.max())
    r /= r.sum()  # probability that x came from each blob
    return r @ (m - x) / v

def drift(x, t):  # probability flow ODE: dx/dt = -beta(t)/2 (x + score)
    return -0.5 * beta(t) * (x + score(x, t))

# Run time backward from 1 to 0.001 in 140 Heun steps, as the demo does.
# There is no randomness: the same start always ends in the same place.
def sample(x):
    ts = 0.001 + 0.999 * np.linspace(1, 0, 141) ** 2
    for t0, t1 in zip(ts[:-1], ts[1:]):
        k1 = drift(x, t0)
        k2 = drift(x + (t1 - t0) * k1, t1)
        x = x + (t1 - t0) * (k1 + k2) / 2
    return x

for start in ([0.5, 0.3], [-0.2, 1.0], [0.5, 0.3]):
    end = sample(np.array(start))
    print(f"noise {start} -> sample ({end[0]:.3f}, {end[1]:.3f})")

t = 0.1: signal kept 0.947, x_t = (1.517, -0.386)
t = 0.3: signal kept 0.630, x_t = (1.177, -0.932)
t = 1.0: signal kept 0.007, x_t = (0.310, -1.200)
noise [0.5, 0.3] -> sample (0.970, 0.030)
noise [-0.2, 1.0] -> sample (-0.899, 0.101)
noise [0.5, 0.3] -> sample (0.970, 0.030)

Running time backward

Destroying structure takes no knowledge at all. Rebuilding it seems to require knowing what the data looks like. The surprising result behind diffusion models is that you need exactly one thing: the scoreScoreThe gradient of the log of a probability density, which points in the direction where data becomes more likely. Diffusion models learn the score of noisy data at every noise level.Open in glossary of the noisy data distribution at every time,

the direction in which noisy data becomes more likely. That is what the arrows draw. With the score in hand, there are two standard ways to run the process in reverse, from $t = 1$ down to $t = 0$:

is the reverse-time stochastic differential equation (a result due to Anderson, 1982, used for generation by Song et al., 2021). It adds fresh randomness at each step, which is the Random (SDE) option. Removing the random term and halving the score term gives the probability flow ODE,

whose paths are deterministic but whose samples, taken together, have exactly the same distribution at every time. That is the Deterministic (ODE) option. In both, the $x$ term undoes the forward shrinking and the score term pulls samples toward likely regions.

What this demo knows that a real model does not

This demo cheats, openly. Its data is a mixture of Gaussians, and adding Gaussian noise to a mixture of Gaussians gives another mixture of Gaussians with wider, shrunken blobs. So $p_t$ and its score have exact formulas, and the arrows you see are the true score.

For images nobody knows $p_t$. A diffusion model therefore trains a neural network to approximate the score. In DDPM the network $\varepsilon_\theta(x_t, t)$ is trained to predict the noise that was mixed in, with a simple loss:

Predicting the noise and estimating the score are the same task in disguise: the best possible noise prediction satisfies $s(x_t, t) = -\,\mathbb{E}[\varepsilon \mid x_t] / \sqrt{1 - \bar\alpha_t}$. Training needs only clean examples and the forward formula, which is why diffusion models are comparatively easy to train.

From dots to pictures

Nothing above depends on the data being two-dimensional. A 512 by 512 color image is 786,432 numbers, and the forward process, the score, and both samplers work the same way in that space. Practical systems add a few ideas on top:

• Latent diffusion (Rombach et al., 2022), used by Stable Diffusion, first compresses images with an autoencoder and runs diffusion in the smaller compressed latent spaceLatent spaceThe space of learned, hidden coordinates a model uses to represent things internally, such as word vectors or a network's hidden layers. Nearby points stand for similar things.Taught in Autoencoders and latent spacesOpen in glossary, which is much cheaper.
• Conditioning feeds the prompt’s text embedding into the noise-prediction network, so the network predicts the noise for “a photo of a cat” rather than for images in general.
• Classifier-free guidanceClassifier-free guidanceA sampling technique for conditional generative models, diffusion or token by token, that compares the model's prediction with and without the prompt and pushes further toward the prompt, trading variety for closer matching.Taught in Diffusion on imagesOpen in glossary (Ho and Salimans, 2022) runs the network with and without the prompt and pushes the result further in the prompt’s direction, trading variety for closer prompt matching.
• Fewer steps. The plain reverse process can take hundreds of steps. Better ODE solvers and distillation methods cut this to a few dozen or even a handful.

The next lesson, Diffusion on images, runs a small trained network of this kind on handwritten digits: it predicts the noise, takes the requested digit as its condition, and supports guidance and a few-step sampler.

Where diffusion stands in 2026

The mathematics above was published between 2020 and 2022, and it still describes how these systems work. What has changed by October 2026, according to the developers’ own reports, is the path from data to noise, the number of steps, and what gets generated.

Straight paths: flow matching. Many recent image models drop the variance-preserving schedule for a simpler one. They walk in a straight line from a data point to its noise,

and train the network, with the same kind of squared-error loss, to predict the direction of that walk: the velocity $\varepsilon - x_0$. This is flow matchingFlow matchingTraining a network to predict the velocity that carries noise to data along chosen paths, then generating by following that velocity from noise. With straight paths it is also called rectified flow.Open in glossary (Lipman et al., 2022) with straight paths, which Liu et al. (2022) call rectified flow. Generating means starting from noise at $t = 1$ and following the predicted velocity back to $t = 0$: a deterministic ODE, like the Deterministic (ODE) option above. The knowledge required is the same. The best possible velocity estimate works out to $-\big(x_t + t\, s(x_t, t)\big) / (1 - t)$, a fixed mix of $x_t$ and the score.

The Stable Diffusion 3 paper (Esser et al., 2024) compared formulations at scale and reported that rectified flow, trained with extra weight on the middle noise levels, beat the established diffusion ones. Since then, open models have documented the same choice: Alibaba’s Qwen-Image “adopt[s] a flow matching training objective” (Qwen team, August 2025), and Black Forest Labs says FLUX.2 “builds on a latent flow matching architecture” (November 2025).

import numpy as np

# The same two blobs as above, now noised along straight paths:
# x_t = (1 - t) x_0 + t * eps, from the data at t = 0 to noise at t = 1.
means, std = np.array([[-1.0, 0.0], [1.0, 0.0]]), 0.1

def clean_estimate(x, t):
    """The exact average of the clean points that could lie under x at time t."""
    m, v = (1 - t) * means, (1 - t) ** 2 * std**2 + t**2
    logits = -np.sum((x - m) ** 2, axis=1) / (2 * v)
    r = np.exp(logits - logits.max())
    r /= r.sum()  # probability that x came from each blob
    return r @ (means + (1 - t) * std**2 / v * (x - m))

# The best velocity estimate is (x - clean estimate) / t, so one Euler step
# from t to t - dt moves the fraction dt / t of the way to the clean estimate.
def sample(x, steps):
    ts = np.linspace(1, 0, steps + 1)
    for t, t_next in zip(ts[:-1], ts[1:]):
        x = x + (t - t_next) / t * (clean_estimate(x, t) - x)
    return x

for steps in (1, 2, 4, 50):
    end = sample(np.array([0.5, 0.3]), steps)
    label = f"{steps} step" + ("s" if steps > 1 else "")
    print(f"{label:>8}: noise [0.5, 0.3] -> ({end[0]:.3f}, {end[1]:.3f})")

  1 step: noise [0.5, 0.3] -> (0.000, 0.000)
 2 steps: noise [0.5, 0.3] -> (0.459, 0.003)
 4 steps: noise [0.5, 0.3] -> (0.964, 0.008)
50 steps: noise [0.5, 0.3] -> (0.969, 0.027)

A single step jumps straight to the clean estimate at $t = 1$, where the noise says nothing about the data, so it lands on the average of the two blobs: a point that is in neither. That is the blur you will see in the next lesson’s clean estimates. On these two blobs, four steps already come close. Image models trained this way still take more: the model cards for Qwen-Image and FLUX.2 [dev] use 50 steps in their examples, and FLUX.2’s card adds that 28 “can be a good trade-off”.

A handful of steps, by distillation. To go further, a second network is trained to do in one to four steps what the first does in dozens. Consistency models (Song et al., 2023) learn to map any point on an ODE path straight to its end. Adversarial diffusion distillation (Sauer et al., 2023) adds a judge network that learns to tell the fast model’s pictures from real ones, and distribution matching distillation (Yin et al., 2023) pushes the fast model’s pictures, taken together, toward the distribution the slow model produces. The Qwen-Image-2.0 report (May 2026) distills a model that takes 40 steps into one that makes comparable pictures with 4 network evaluations.

Video, and sound with it. Video is a stack of frames, so the same process runs on compressed video. OpenAI’s Sora (2024) did this with a transformer. Google’s Veo 3 report (May 2025) says the model “uses latent diffusion, in which the diffusion process is applied jointly to the temporal audio latents, and the spatio-temporal video latents”, with “a transformer-based denoising network” doing the denoising, so picture and sound are generated together rather than one after the other. Open video models such as Alibaba’s Wan, released at 1.3 and 14 billion parameters, are “built upon the mainstream diffusion transformer paradigm” (Wan team, 2025).

Text, experimentally. Words are not continuous numbers, so text diffusion replaces Gaussian noise with masking: the forward process hides tokens, a transformer learns to fill them back in, and generation unmasks many tokens at each step instead of writing one at a time. LLaDA, an 8-billion-parameter model trained this way, was “competitive with strong LLMs like LLaMA3 8B in in-context learning” (Nie et al., 2025). As of October 2026 Google describes its Gemini Diffusion as “an experimental demo” (Google DeepMind).