Thinking longer

A language model produces its answer one tokenTokenThe unit of text a language model reads and writes. A token can be a whole word, part of a word, a single character, or punctuation, often including a leading space.Taught in TokensOpen in glossary at a time, and each new token passes through the same fixed stack of layers, so the computation available per token is roughly fixed (attention grows somewhat with the length of the text, but the depth does not). Ask a hard multi-step question and demand the answer immediately, and the model has to do all the work in the few tokens before it commits. Researchers found several ways to give models more room, all of which spend more computation at answer time. This family of ideas is called test-time computeTest-time computeComputation a model spends while answering, rather than during training, such as writing out reasoning, sampling several answers, or searching over candidates. More of it often improves accuracy on hard problems.Open in glossary.

Writing out the steps

The first idea is to let the model write its reasoning before the answer. Wei et al. (2022) showed that including a few worked examples with step-by-step solutions in the prompt, which they called chain-of-thoughtChain of thoughtIntermediate reasoning steps a language model writes before its final answer. Prompting for them, or training models to produce them, often improves results on multi-step problems.Open in glossary prompting, sharply improved large models on math word problems and other multi-step tasks. Kojima et al. (2022) found that even appending “Let’s think step by step” to a question helped.

One common explanation is that the written steps act as working memory. Every token the model writes becomes part of its input for the next token, so intermediate results do not have to be held inside a single forward pass. This explanation fits the evidence but is not the whole story, and why chain of thought helps on some tasks and not others is still an active research question.

Asking several times and voting

If you sample with a temperatureTemperatureA setting for how adventurous a model's choices are. Low temperature sticks to the likeliest options; high temperature spreads choices toward less likely ones. Technically, it divides the logits before softmax.Taught in Predicting the next tokenOpen in glossary above zero, the same question can produce different reasoning paths and different final answers. Self-consistencySelf-consistencySampling several reasoning paths for the same question and returning the most common final answer. It helps when the correct answer is the model's most likely answer.Open in glossary (Wang et al., 2022) samples several chains of thought and returns the most common final answer.

How much does that help? If you treat each sample as an independent draw that is right with probability $p$, the answer can be computed exactly. With two possible answers and an odd number of samples $n$, the vote is right when more than half of the samples are:

With more possible answers the same idea holds, summing over every possible tally of answers (a multinomial), and splitting ties evenly. The demo does exactly that.

from itertools import product
from math import comb, factorial, prod

def majority(p, n):
    # Two answers, odd n: the vote is right when more than half the samples are.
    return sum(comb(n, k) * p**k * (1 - p) ** (n - k) for k in range(n // 2 + 1, n + 1))

for p in (0.6, 0.4):
    print(f"2 answers, p = {p}:", ", ".join(f"n={n}: {majority(p, n):.3f}" for n in (1, 5, 25)))

def plurality(probs, n):
    # Sum over every tally of n answers (answer 0 is correct); split ties evenly.
    total = 0.0
    for counts in product(range(n + 1), repeat=len(probs) - 1):
        if sum(counts) > n:
            continue
        counts = (*counts, n - sum(counts))
        top = max(counts)
        if counts[0] == top:
            ways = factorial(n) // prod(factorial(c) for c in counts)
            total += ways * prod(q**c for q, c in zip(probs, counts)) / counts.count(top)
    return total

# The demo's four-answer setups. Answer 0 is correct.
spread = [0.4, 0.2, 0.2, 0.2]  # wrong answers spread out
tempting = [0.3, 0.49, 0.105, 0.105]  # one tempting wrong answer
for name, probs in (("4, spread", spread), ("4, one tempting", tempting)):
    accuracy = (f"n={n}: {plurality(probs, n):.3f}" for n in (1, 9, 41))
    print(f"{name}, p = {probs[0]}:", ", ".join(accuracy))

2 answers, p = 0.6: n=1: 0.600, n=5: 0.683, n=25: 0.846
2 answers, p = 0.4: n=1: 0.400, n=5: 0.317, n=25: 0.154
4, spread, p = 0.4: n=1: 0.400, n=9: 0.599, n=41: 0.894
4, one tempting, p = 0.3: n=1: 0.300, n=9: 0.244, n=41: 0.083

The rule behind every curve: voting converges on the most likely answer. With enough samples, it helps exactly when the correct answer is the single most common output, whether or not that answer is right more than half the time. It cannot rescue a model whose most common answer is wrong.

Models trained to reason

Prompting and voting work on any model. A newer approach trains the model itself to produce long reasoning before answering, giving what are called reasoning modelsReasoning modelA language model trained, often with rewards for answers that can be checked, to write long reasoning before it answers. More thinking often helps on hard problems, at the cost of more tokens, time, and energy.Open in glossary. OpenAI’s o1 (previewed in September 2024, released in full that December) and DeepSeek-R1 (January 2025) are prominent examples. DeepSeek’s report describes reinforcement learning in which the model is rewarded for reaching correct answers on problems that can be checked automatically, such as math with known answers and code with tests. A variant trained with reinforcement learning alone (R1-Zero) already learned long reasoning; the released R1 added supervised fine-tuning stages before and between rounds of reinforcement learning to make its output more readable. Trained this way, models learned to write long chains that include checking their work and backtracking. (The reinforcement learning track explains rewards that can be checked automatically and the policy gradient methods behind this training.)

OpenAI also reported that accuracy kept rising as the model was allowed to reason for longer, roughly in proportion to the logarithm of the extra computation over the range they tested. This is described as scaling with test-time compute, alongside the familiar scaling with training compute. The cost is real: long reasoning can mean thousands of extra tokens per answer, which costs time and money.

A small reasoning model, recorded

DeepSeek also released six smaller distilledKnowledge distillationTraining a smaller student model to reproduce a larger teacher model's outputs, so the student learns from the teacher's predictions instead of, or as well as, human labels.Taught in Depth from a single imageOpen in glossary models: ordinary open models fine-tuned on about 800,000 samples curated with R1, according to the model card. The smallest, DeepSeek-R1-Distill-Qwen-1.5B, is built on Qwen2.5-Math-1.5B and runs on a laptop. We gave it twelve short problems with known answers and recorded 16 reasoning traces for each, at the settings its model card recommends and evaluates with (temperature 0.6, top-p 0.95, and here at most 4,096 tokens). To see what the thinking buys, each trace was also cut short at budgets from 0 to 4,096 thinking tokens, and the model was made to answer at that point. Cutting thinking short this way is one half of budget forcingBudget forcingControlling how long a reasoning model thinks. Adding the end-of-thinking marker after a set number of tokens cuts it short; holding the marker back and appending "Wait" extends it. From the s1 paper (Muennighoff and colleagues, 2025).Open in glossary (Muennighoff et al., 2025); the other half makes a model think longer by holding back the end-of-thinking marker and appending “Wait”.

On these problems thinking helps a great deal, but not smoothly. On the Sisters puzzle, traces cut off after 128 tokens had often just reasoned that if each brother has 2 sisters, there are 2 sisters in total, counting every girl in the family, so the forced answer is 2; given room to finish, most of them subtract Sally. Thinking can also fail to end: 12 of the 192 sampled traces ran out of tokens while still thinking, and so did 3 of the 12 greedy traces (temperature 0), two of them repeating the same lines over and over. That is why the model card recommends a temperature between 0.5 and 0.7.

The recording also shows the correlation the note above warns about. At a 512-token budget one sample is right 81% of the time. If every problem were a coin weighted 81% toward the right answer, a vote of 15 samples would be right 99.7% of the time. Drawing 15 votes from each problem’s own recorded answers gives 89.5% on average: the mistakes cluster on a few problems, and on Sisters the most common answer at that budget is wrong, so a bigger vote only picks that wrong answer more reliably.

These are short puzzles that the model mostly solves. On harder problems the same model does much worse: DeepSeek reports it right 28.9% of the time on the AIME 2024 competition problems, against 83.9% on the MATH-500 set.

Where reasoning models stand in 2026

Since o1 and R1, long reasoning has become a standard feature of large models. This section is dated October 2026; the field moves fast, so check what is current.

Thinking is a setting. In April 2025, Qwen3 let one open model switch between a thinking and a non-thinking mode. As of October 2026, the large providers let a developer choose an effort level rather than a fixed number of thinking tokens. OpenAI’s reasoning models take an effort setting whose allowed values, from none to max, depend on the model (OpenAI). Anthropic’s current models use adaptive thinking: the model decides whether and how much to think on each request, steered by an effort setting (Anthropic). Both bill thinking tokens as output tokens, and both count them in the context window. Neither shows you the raw reasoning: OpenAI does not expose the reasoning tokens, and Anthropic’s documentation says the text in a thinking block is “never the raw chain of thought” but a summary of it. Open models, like the one recorded above, show everything they write.

What it reached. In July 2025, an advanced version of Google’s Gemini Deep Think scored 35 of 42 points on the problems of the 2025 International Mathematical Olympiad, a gold-medal standard, writing proofs in natural language that the competition’s coordinators graded (Google DeepMind). OpenAI reported the same score for an experimental reasoning model, graded by three former medalists instead.

What it cost to train. DeepSeek’s R1 paper passed peer review and appeared in Nature in September 2025 (DeepSeek-AI, 2025). Its supplementary information puts the reinforcement learning and data preparation that turned the DeepSeek-V3 base model into R1-Zero and R1 at about 147,000 H800 GPU hours, $294,000 at an assumed $2 an hour. That is on top of the base model, whose final training run DeepSeek put at $5.6 million (Compute and energy explains what such figures leave out).

Limits that have been measured.

• Training mostly sharpens what was there. Yue and colleagues (NeurIPS 2025) compared models trained with checkable rewards against the base models they started from. With one try per problem, the trained models won. With many tries, the base models solved more problems (pass@k with large k). The authors conclude that, in the setups they tested, this training made the model pick good reasoning it could already produce rather than find new kinds, while distilling from a stronger teacher could add new patterns.
• More thinking can hurt. Gema and colleagues (2025) built tasks, such as simple counting questions padded with distracting details, on which reasoning models did worse the longer they reasoned. The Sisters puzzle above shows a related effect: the recorded model answered it correctly more often with no thinking than with any budget of thinking.
• Whether accuracy collapses on long puzzles is disputed. Shojaee and colleagues (Apple, 2025) reported reasoning models failing completely beyond a certain puzzle size, and, past a point, writing less reasoning as puzzles grew harder. A reply argued that some of those failures came from answers longer than the models were allowed to write, and from puzzle instances that had no solution.
• The written reasoning is still not a faithful record. Chen and colleagues (Anthropic, 2025) slipped hints into questions and checked which answers the hints changed. In those cases, Claude 3.7 Sonnet mentioned the hint in its reasoning 25% of the time and DeepSeek-R1 39% of the time, averaged over the kinds of hint. The misconception above applies to reasoning models too.

Thinking also costs energy. A 2026 estimate by Microsoft researchers puts a typical query to a frontier-scale model at a median of 0.31 Wh, and a reasoning-style answer 15 times longer at 3.91 Wh (Oviedo and colleagues, 2026).