Base rates

A thousand people take a medical test. See why a positive result from a good test can still mean a small chance of being sick.

BeginnerExplained in Probability and Bayes' rule

Base rates

A thousand people take a test for a condition. Filled marks tested positive; faded marks tested negative.

  • Sick (10)
  • Healthy (990)

Everyone who tested positive: 98 people

9 sick 89 healthy

Counts are rounded to whole people; the percentages use exact probabilities.

1%
90%
91%
1
Chance a positive is really sick9.2%
Chance before any test1%
A positive multiplies the odds by10

Challenges

0 of 3 complete

  1. Keep the starting 1% rate and test. Find how many positive results in a row it takes before the person is more likely sick than not. (not yet)
  2. Make the test right at least 99% of the time for sick and for healthy people, then find a base rate where a single positive is still more likely a false alarm. (not yet)
  3. With the starting test, find the base rate at which a single positive result is a coin flip (between 48% and 52%). (not yet)

Try this

  • Before touching anything, guess the chance that someone who tests positive is sick. Then read it off the demo.
  • Drag the base rate up to 20%. The test has not changed, yet a positive result now means something very different.
  • Raise Positive results in a row to 2 and 3 with the starting settings. Each positive multiplies the odds by the same factor.
  • Set specificity to 99.9% and watch the false alarms in the grid nearly vanish.
Bayes' rule with counts and with oddsPython with no libraries
prevalence = 0.01   # 1% of people have the condition
sensitivity = 0.90  # the test catches 90% of sick people
specificity = 0.91  # and clears 91% of healthy people

# Natural frequencies: the same numbers as counts out of 1000 people.
people = 1000
sick = people * prevalence
healthy = people - sick
true_pos = sick * sensitivity
false_pos = healthy * (1 - specificity)
print(f"{true_pos:.1f} sick and {false_pos:.1f} healthy people test positive")

# Bayes' rule: the share of positives who are really sick.
print(f"P(sick | positive) = {true_pos / (true_pos + false_pos):.3f}")

# Odds form: each positive multiplies the odds by the likelihood ratio
# (assuming repeat tests make independent mistakes).
odds = prevalence / (1 - prevalence)
ratio = sensitivity / (1 - specificity)
for n in range(1, 4):
    odds *= ratio
    print(f"positives in a row: {n}, P(sick) = {odds / (1 + odds):.3f}")
9.0 sick and 89.1 healthy people test positive
P(sick | positive) = 0.092
positives in a row: 1, P(sick) = 0.092
positives in a row: 2, P(sick) = 0.503
positives in a row: 3, P(sick) = 0.910

Try "embedding", "softmax", "overfitting", or "backpropagation".