Evaluation: what a score means

Every new model arrives with a table of scores: 86.4% on one test, 71% on another, first place on a leaderboard. Those numbers are useful, and they are easy to read too much into. This lesson is about what a score actually measures, how much it can move by chance, and three ways scores are computed for modern models.

A benchmark is a sample

A benchmarkBenchmarkA fixed set of test questions or tasks with a scoring rule, used to compare models. A score is an estimate from a sample of questions, so it comes with noise.Open in glossary is a fixed set of questions or tasks with a scoring rule. MMLU, for example, has multiple-choice questions from 57 subjects, from law to physics (Hendrycks and colleagues, 2021). HumanEval has 164 programming problems, each checked by running unit tests (Chen and colleagues, 2021).

What you want to know is how well a model does on the kind of task the benchmark represents. The benchmark’s questions are one sample of that kind of task. Swap in a different sample of similar questions and the score would change, the same way a poll changes when you ask different people.

For an accuracy $p$ measured on $n$ independent questions, the standard error and the usual 95% confidence intervalConfidence intervalA range around a measured number that shows how much it could move by chance. For an accuracy p on n questions, the usual 95% interval (the normal approximation) is about p plus or minus 1.96 times the square root of p(1 - p)/n.Open in glossary, based on the normal approximation, are

This simple interval (often called the Wald interval) works well when $np$ and $n(1-p)$ are both at least about 10; for tiny benchmarks or scores near 0% or 100%, methods such as the Wilson interval are more reliable. On HumanEval’s 164 problems, a score near 50% has a 95% interval of about $\pm 7.7$ points. That is wider than the gap between many models that get compared on it.

Which model is better?

Below, two models with known true accuracies, 3 points apart, take the same benchmark. Run it and see what you would have reported.

import numpy as np
from math import lgamma, log

def half_width(score, n):
    """95% interval half-width: 1.96 standard errors of an accuracy."""
    return 1.96 * np.sqrt(score * (1 - score) / n)

def binomial(n, p):
    """Probability of each score 0..n when each answer is right with probability p."""
    k = np.arange(n + 1)
    logc = np.array([lgamma(n + 1) - lgamma(i + 1) - lgamma(n - i + 1) for i in k])
    return np.exp(logc + k * log(p) + (n - k) * log(1 - p))

def upset(n, weak=0.70, strong=0.73):
    """Exact chance that the weaker model scores at least as high as the stronger."""
    pw, ps = binomial(n, weak), binomial(n, strong)
    weak_at_least = np.cumsum(pw[::-1])[::-1]  # P(weak score >= s)
    return np.sum(ps * weak_at_least)

for n in [100, 164, 1000, 2000]:
    print(f"{n:5d} questions: 95% interval at 50% is +/-{100 * half_width(0.5, n):.1f} points, "
          f"weaker model ties or wins {upset(n):.1%} of the time")

  100 questions: 95% interval at 50% is +/-9.8 points, weaker model ties or wins 34.7% of the time
  164 questions: 95% interval at 50% is +/-7.7 points, weaker model ties or wins 29.4% of the time
 1000 questions: 95% interval at 50% is +/-3.1 points, weaker model ties or wins 7.2% of the time
 2000 questions: 95% interval at 50% is +/-2.2 points, weaker model ties or wins 1.9% of the time

With 100 questions, the weaker model ties or beats the stronger one about 35% of the time. That chance only falls below 5% at about 1,300 questions; on the demo’s slider, 2,000 is the first size that gets there. The interval shrinks with the square root of the number of questions, so halving it takes four times as many.

Contamination: when the test was in the training data

A benchmark only measures skill on new problems if the problems are new to the model. Large language models are trained on huge scrapes of the web, and benchmark questions get posted online, discussed, and copied. When test questions or their answers end up in the training data, part of the score measures memory. This is called data contaminationData contaminationWhen test questions, or their answers, end up in a model's training data, so a benchmark score partly measures memory instead of skill.Open in glossary.

Developers try to detect and remove it. The GPT-3 authors (Brown and colleagues, 2020) searched their training data for word sequences that also appeared in their benchmarks (13 words long for most benchmarks, as short as 8 for benchmarks with short examples), then compared each full score with the score on a “clean” subset of questions that had no overlap. BIG-bench, a large collection of tasks, puts a unique “canary” string in every task file so that anyone building a training set can find and filter it out.

Another test is to write new questions. Zhang and colleagues (2024) wrote GSM1k, 1,250 new grade-school math problems matched in style and difficulty to the widely used GSM8K. Some model families scored lower on the new problems across almost all their sizes, by up to 8 points in the published version of the study (the first preprint reported up to 13), a sign that they had partly memorized the old ones. Several frontier models showed little sign of this.

When benchmarks wear out

A benchmark is useful while models differ on it. Once the best models score near the top, the gaps between them shrink inside the error bars, and many of the remaining misses are questions with a wrong answer key. Benchmark makers respond with harder tests, and those wear out too. Two examples, dated as of October 2026:

Humanity’s Last Exam (Center for AI Safety, Scale AI, and contributors, 2025, published in Nature in 2026) gathered questions from experts in over a hundred subjects and kept only those that the best models of the time got wrong, so its low starting scores were partly by design. At release in January 2025, no model tested reached 10%, and all were badly calibrated, often giving wrong answers with high stated confidence. After a year of reviews and corrections, its makers released a cleaned subset of 1,000 questions, HLE-Diamond, in September 2026. The best score they reported was 59.9% without tools and 82.9% with web search and code execution (HLE-Diamond). The first section of this lesson applies: with 1,000 questions the 95% interval is about ±3 points, so the second and third places, 54.6% and 53.2%, are within the noise of each other. And a score belongs to a setup, not only to a model: the six models also tested with tools scored 19 to 32 points higher with them.

Time horizons. METR, a nonprofit research group, measures how long a task an AI agent can finish. It times skilled people on software and research tasks, runs agents on the same tasks, and reports each model’s time horizonTime horizonMETR's measure of what an AI agent can do: the length of task, in the time a skilled person needs for it, that the agent completes with 50% success. It depends on the tasks chosen and how long people take.Open in glossary: the length of task, in human working time, that the model completes with 50% success (Kwa and colleagues, 2025). In its January 2026 revision, with 228 tasks, the time horizon of the best models had doubled about every 131 days since 2023 (METR, January 2026). By May 2026, METR wrote that the most capable agents had “essentially saturated” those tasks: its estimate for the best public model was about 12 hours, with a range from 5 to 61 hours, and the task set cannot reliably measure horizons above 16 hours. It also found that at least 16% of successful runs on its tasks over 8 hours long were illegitimate on review; in one, an agent modified the task’s simulator to make the task trivial (METR, May 2026). That is reward hacking inside an evaluation, and it is why careful evaluators read transcripts rather than trusting a pass or fail.

pass@k: when a model gets several tries

Code models can generate many attempts, and tests can check each one. pass@kpass@kThe chance that at least one of k attempts at a problem is correct, averaged over problems. Usually estimated from n ≥ k samples with the unbiased formula 1 - C(n - c, k) / C(n, k).Open in glossary is the chance that at least one of $k$ attempts at a problem is correct, averaged over problems. For a problem the model solves with probability $p$ per attempt, it is $1 - (1-p)^k$.

You never know $p$. You draw $n \ge k$ samples per problem and count the $c$ that pass. The tempting shortcut is to plug $c/n$ in for $p$:

Chen and colleagues (2021) pointed out that this is biased, and used an unbiased estimator instead. Pick $k$ of your $n$ samples at random. The chance that none of them is correct is the number of ways to pick $k$ from the $n - c$ failures, divided by the number of ways to pick $k$ from all $n$:

Its average over all possible draws is exactly the true pass@k. They generated $n = 200$ samples per problem and reported $k$ up to 100.

import numpy as np
from math import comb

def pass_at_k(n, c, k):
    """Unbiased estimator from Chen et al. (2021): 1 - C(n-c, k) / C(n, k)."""
    if n - c < k:
        return 1.0
    return 1.0 - np.prod(1.0 - k / np.arange(n - c + 1, n + 1))

def plug_in(n, c, k):
    """The tempting shortcut: estimate p = c/n, then 1 - (1 - p)^k."""
    return 1.0 - (1.0 - c / n) ** k

# One problem the model solves 10% of the time, n = 10 samples.
p, n = 0.1, 10
chance = [comb(n, c) * p**c * (1 - p) ** (n - c) for c in range(n + 1)]
print(" k   true   unbiased (avg)   plug-in (avg)")
for k in [1, 5, 10]:
    true = 1 - (1 - p) ** k
    unbiased = sum(w * pass_at_k(n, c, k) for c, w in enumerate(chance))
    naive = sum(w * plug_in(n, c, k) for c, w in enumerate(chance))
    print(f"{k:2d}  {true:.3f}       {unbiased:.3f}           {naive:.3f}")

 k   true   unbiased (avg)   plug-in (avg)
 1  0.100       0.100           0.100
 5  0.410       0.410           0.349
10  0.651       0.651           0.494

The plug-in estimate is exact at $k = 1$ and too low for every larger $k$, worst when $k$ is close to $n$. On the demo’s 20 problems with 10 samples each, its average for pass@10 is 62.8% against a true 71.9%. The reason is the curve: $1 - (1-p)^k$ bends, so a noisy $c/n$ that is too high gains less than one that is too low loses. The unbiased estimator still varies from draw to draw. Unbiased means right on average, not right every time.

pass^k: when every try has to work

For an agent that serves customers, one success in k tries is not enough; you want it to work every time. τ-bench (Yao and colleagues, 2024) introduced pass^k (“pass hat k”): the chance that all $k$ attempts succeed, $p^k$ for a problem solved with probability $p$, averaged over problems. From $n$ samples with $c$ successes, its unbiased estimate is $\binom{c}{k} / \binom{n}{k}$, by the same counting of subsets as above. The two measures move apart as $k$ grows. For a problem solved 80% of the time, pass@8 is above 99.99%, while pass^8 is $0.8^8 \approx 17\%$. Models that use tools gives the numbers τ-bench measured.

Ratings from pairwise votes

Some qualities, like how helpful an answer is, have no answer key. Chatbot Arena (Chiang and colleagues, 2024) asks people instead: a visitor types a prompt, two anonymous models answer, and the visitor votes for the better one. The votes become ratings on the EloElo ratingA rating scale from chess where a 400-point gap means 10 to 1 odds of winning. Online Elo nudges two ratings after every game; leaderboards like Chatbot Arena report ratings on this scale.Open in glossary scale from chess, where model $A$ is expected to beat model $B$ with probability

Online Elo updates both ratings after each vote, by $R_A \leftarrow R_A + K\,(S_A - E_A)$, where $S_A$ is 1 for a win and 0 for a loss, and $B$ moves by the opposite amount. Chatbot Arena’s code uses $K = 4$. Recent votes count more than old ones, so the ratings depend on the order the votes arrived in.

Bradley-Terry uses the same win probabilities but fits all the ratings at once, choosing the ones that make the observed votes most likely. It is the same Bradley-Terry modelBradley-Terry modelA model of pairwise comparisons: each item has a score, and the chance that A beats B is the sigmoid of A's score minus B's. Reward models for RLHF are trained with it.Taught in Rewarding LLMs, and reward hackingOpen in glossary that trains reward models in the RLHF lesson. Order does not matter, and resampling the votes many times (a bootstrap) gives an interval for each rating. Chatbot Arena moved from online Elo to Bradley-Terry in December 2023, saying it gave significantly more stable ratings and more precise confidence intervals.

import numpy as np

# Each vote is (winner, loser) among three models: 0, 1, 2.
votes = [(0, 1), (0, 2), (1, 2), (0, 1), (2, 1), (1, 0), (0, 2), (1, 2), (0, 1), (2, 0)]

def expected(ra, rb):
    """Elo: chance that A beats B."""
    return 1 / (1 + 10 ** ((rb - ra) / 400))

def online_elo(votes, k=32):
    r = np.full(3, 1000.0)
    for w, l in votes:
        d = k * (1 - expected(r[w], r[l]))
        r[w] += d
        r[l] -= d
    return r

def bradley_terry(votes, iters=200):
    """Maximum likelihood with the classic iterative update (Zermelo; Hunter 2004)."""
    wins = np.zeros((3, 3))
    for w, l in votes:
        wins[w, l] += 1
    games = wins + wins.T
    s = np.ones(3)  # strengths; P(i beats j) = s_i / (s_i + s_j)
    for _ in range(iters):
        s = wins.sum(1) / (games / (s[:, None] + s[None, :])).sum(1)
        s /= np.exp(np.log(s).mean())
    return 1000 + 400 * np.log10(s)  # Elo scale, mean log strength at 1000

print("online Elo, original order:", np.round(online_elo(votes)).astype(int).tolist())
print("online Elo, reversed order:", np.round(online_elo(votes[::-1])).astype(int).tolist())
print("Bradley-Terry, either order:", np.round(bradley_terry(votes)).astype(int).tolist(),
      np.round(bradley_terry(votes[::-1])).astype(int).tolist())

online Elo, original order: [1034, 989, 977]
online Elo, reversed order: [1045, 986, 969]
Bradley-Terry, either order: [1109, 974, 917] [1109, 974, 917]

In the demo, five models have true ratings from 900 to 1090, and votes are simulated from those ratings. With Bradley-Terry, the ratings are within 30 points of the truth by 1,000 votes, but the intervals only stop overlapping after a couple of thousand votes, because the closest models are just 30 points apart. Online Elo with $K = 4$ moves slowly: after 100 votes its ratings are all still within 50 points of 1000, while Bradley-Terry already spreads them over more than 250, more than the true 190, because estimates from so few votes are noisy. And the same votes in a different order give online Elo ratings that differ by more than 10 points once there are a few hundred votes.

An arena measures what its voters prefer. If they prefer longer or more heavily formatted answers, a model that writes them climbs, whatever its substance. Turn on the length-favoring voters in the demo, who pick the longer answer 70% of the time between two equally good ones, and after a few thousand votes the truly weakest model ranks second. Chatbot Arena later added a style-controlled ranking that adjusts for answer length and formatting, a reminder that every evaluation, like every reward, measures something specific.

Which models get rated, and how often, matters too. Singh and colleagues (2025) found that some providers tested many private versions of a model before release and could publish only the best, 27 from Meta before Llama 4, and that Google and OpenAI had each received about a fifth of all the arena’s data, while 83 open models together received under a third. Reporting the best of many noisy ratings inflates the winner, the same selection effect as reporting your best of several benchmark runs. The arena’s operators replied that the effect was smaller than claimed and fades as new votes arrive, and changed their policy: when more than 10 versions are tested in parallel before a release, the released model’s score is marked provisional until 2,000 new votes arrive (Arena, 2025). The project, later called LMArena, has been called simply Arena since January 2026.

Reading a score

A few questions turn a score into evidence:

• How many questions, and what interval? A gap smaller than the interval is not a finding.
• Could the model have seen the test? Fresh or private test sets are more trustworthy than famous old ones.
• How many tries, and how was the score estimated? pass@1 and pass@100 are very different claims, and pass^k is a third.
• Under what setup? Tools, the agent’s harness, and the thinking effort can each move a score by more than the gap between models.
• Is the benchmark worn out, and how were the passes earned? Near the top of a benchmark, the remaining misses may be broken questions, and an agent can pass a check without doing the task.
• What does it measure? A multiple-choice exam, a unit test, and a vote all measure different things, and none of them is “intelligence”.

Evaluations also matter for safety, where the question is what a model could do that would be harmful. The lesson on alignment covers how those evaluations and red teaming are used.