Text to speech

Screen readers, navigation apps, and audiobook narrators all read text aloud. For about two decades the best-sounding systems stitched together tiny pieces of recorded speech; the five speakers in the speech-to-text lab come from CMU ARCTIC, a set of recordings made in 2003 for exactly that kind of system. Today’s systems are neural networks that generate the sound. This lesson runs one in your browser and looks at what it makes.

The classic pipeline

Text to speechText to speechTurning written text into spoken audio. Neural systems predict an intermediate representation of the sound, such as a spectrogram or a learned code, and a decoder or vocoder turns it into a waveform.Open in glossary has to solve two problems: what the speech should sound like, and how to produce the 44,100 or so numbers per second that make it audible. A common neural recipe splits them. An acoustic model reads the text and predicts a mel spectrogram, the picture from How computers hear. A second network, the vocoderVocoderThe part of a speech system that turns a compact description of sound, such as a mel spectrogram or a learned code, into an audio waveform, filling in detail the description leaves out, such as phase.Open in glossary, turns that spectrogram into a waveform.

Tacotron 2 (Google, 2017) is the classic example: a sequence-to-sequence network mapped characters to mel spectrograms, and a modified WaveNet, a network from 2016 that generates audio one sample at a time, served as the vocoder. Later vocoders such as HiFi-GAN (2020) did the same job much faster.

Why not stop at the spectrogram? Because a spectrogram has no phase. Many waveforms share one spectrogram, and most of them sound buzzy or smeared. The vocoder’s job is to produce one that sounds natural.

What this lab runs: Supertonic 2

The lab uses Supertonic 2, a 66-million-parameter model released by Supertone in January 2026. Its design, described in Supertone’s paper on SupertonicTTS (Kim and colleagues, 2025, a smaller 44-million-parameter model), replaces the hand-made spectrogram in the middle with a code the model learned. The files the browser downloads follow the same plan, in three networks:

1. Text encoder. It reads the text as plain characters, with no separate step that converts spelling to phonemes, together with a voice. A voice is a style vector, 12,928 numbers that the model ships for each of its ten voices (F1 to F5 and M1 to M5). It also predicts how long the whole utterance should last. The lab’s speed setting simply divides that length.
2. Denoiser. It starts from random noise, one block of 144 numbers for every 70 ms of speech, and refines it step by step toward a code for speech that fits the text and voice. It looks at the text through cross-attention, which is how it works out which sound goes where. This is flow matching, a close relative of the diffusion modelsDiffusion modelA generative model that learns to reverse a gradual noising process, so it can turn pure random noise into new data such as images, step by step.Taught in Diffusion: from noise to dataOpen in glossary in the frontiers track.
3. Voice decoder. It turns the finished code into a waveform at 44,100 samples per second. It is the second half of an autoencoderAutoencoderA network trained to rebuild its own input after squeezing it through a narrow middle layer. The encoder half compresses; the decoder half reconstructs; the middle layer's values are a learned code.Taught in Autoencoders and latent spacesOpen in glossary trained to compress and rebuild speech; its other half, used only in training, read mel spectrograms. The paper’s authors describe the autoencoder as “a neural vocoder with a compact latent space in its middle”.

So the model never writes a spectrogram. The spectrogram in the lab is computed from its output with the same code as the spectrogram lab, the same way you would analyze any recording.

import numpy as np

# A toy flow-matching sampler. The "data" are two possible values, -1 and +1
# (think: two ways to say a sound). Samples start as Gaussian noise at t = 0 and
# follow a velocity field to t = 1. For this data the ideal velocity is exact:
# v(x, t) = (E[data | x at time t] - x) / (1 - t), with E[...] = tanh(x t / (1 - t)^2).
def velocity(x, t):
    return (np.tanh(x * t / (1 - t) ** 2) - x) / (1 - t)

rng = np.random.default_rng(0)
start = rng.standard_normal(10_000)
for steps in (1, 2, 5, 10, 50):
    x = start.copy()
    for k in range(steps):                     # Euler steps from t = 0 to t = 1
        t = k / steps
        x = x + velocity(x, t) / steps
    near = np.mean(np.abs(np.abs(x) - 1) < 0.1)
    print(f"{steps:2d} steps: mean |x| = {np.mean(np.abs(x)):.3f}, within 0.1 of -1 or +1: {near:.0%}")

 1 steps: mean |x| = 0.000, within 0.1 of -1 or +1: 0%
 2 steps: mean |x| = 0.559, within 0.1 of -1 or +1: 14%
 5 steps: mean |x| = 0.986, within 0.1 of -1 or +1: 97%
10 steps: mean |x| = 1.000, within 0.1 of -1 or +1: 100%
50 steps: mean |x| = 1.000, within 0.1 of -1 or +1: 100%

What you saw

Pitch and voices. A voiced sound repeats at its pitch, and anything that repeats $f$ times a second contains the frequencies $f, 2f, 3f, \ldots$, the evenly spaced lines in the spectrogram. The lab measures pitch with the YIN method (it finds the shortest delay at which the waveform repeats) and draws it in orange along the lowest line. In our measurements the female voices sat between about 150 and 240 Hz and the male voices between about 90 and 160 Hz, depending on the sentence. The style vector changes how the voice sounds, not what it says.

Steps. The code example shows the idea with two possible “sounds”, -1 and +1. From noise, the best single move points to their average, 0, so one step puts every sample there; a few more steps let each sample commit to one value. Real speech has countless possible codes, and the one-step result is a blurred average of them. Supertone benchmarks Supertonic 2 at just 2 and 5 steps, and its one-step speech is usually understandable. We tested this by passing its speech through Whisper (three sentences, three different starting noises, voice F1): one step averaged about 12 word errors per 100, two steps 7, five steps 5, and ten steps 6. Past a few steps, more did not help.

Speed. The denoiser fills whatever length the duration predictor allows, so doubling the speed packs the same words into half the time. At 2x, Whisper got at least half the words wrong in each of our three sentences (voice F1).

What it gets wrong. Supertonic reads characters, so it has to infer pronunciation from spelling alone. In our tests, Whisper heard “The quick brown fox” as “The quick-bound fox” when spoken by voice F1, though seven of the other nine voices came through intact. Names, numbers, abbreviations, and words spelled the same but pronounced differently are where systems without a pronunciation dictionary tend to slip; listen for them.