Confident, and wrong

A single neuron learned from data in the dashed box on the left. Slide the solid box to test it on new data from elsewhere.

Where the new data comes fromInside the training area

Show the true rule

Show the training data

Accuracy where it learned99%

Accuracy on the new data100%

How sure it sounds on the new data97%

Challenges

0 of 3 complete

Make it fail: find new data where the model gets 1 in 4 points or more wrong (accuracy 75% or lower). (not yet)
Catch it bluffing: find new data where accuracy is 70% or lower, yet it still sounds at least 95% sure. (not yet)
Find out why: with the new data far away, turn on Show the true rule. (not yet)

The solid line is the model's boundary; the dashed line is the true rule. Ringed points are new examples the model gets wrong. "How sure it sounds" averages the model's probability for whichever answer it gives, from 50% (a coin flip) to 100%.

Try this

Slide the new data from the left edge to the right edge and watch the two accuracy numbers part ways.
Turn on Show the true rule to see why: the truth curves, but the model only ever saw the straight part.
Watch the confidence number as accuracy falls. Does the model warn you?

Optional: the same idea in codePython with NumPy

import numpy as np

rng = np.random.default_rng(1)

def make_points(n, x_from, x_to):
    across = rng.uniform(x_from, x_to, n)
    up = rng.uniform(0.03, 0.97, n)
    # The true rule: orange (1) above the curve up = 0.3 + 0.6 * across**2.
    labels = (up > 0.3 + 0.6 * across**2).astype(float)
    inputs = np.column_stack([across, up, np.ones(n)])  # the 1 is for the bias
    return inputs, labels

# Train one neuron only on the left part, where the curve is nearly straight.
inputs, labels = make_points(90, 0.02, 0.4)
weights = np.zeros(3)
for _ in range(4000):
    chance = 1 / (1 + np.exp(-inputs @ weights))  # its chance of orange
    weights -= 4 * inputs.T @ (chance - labels) / len(labels)  # one nudge

# Test it on new points near its training data, and far from it
# (the far points come from where the demo's slider ends).
for name, x_from, x_to in [("Near", 0.02, 0.4), ("Far", 0.78, 1.0)]:
    inputs, labels = make_points(400, x_from, x_to)
    chance = 1 / (1 + np.exp(-inputs @ weights))
    right = np.mean((chance >= 0.5) == labels)
    sure = np.mean(np.maximum(chance, 1 - chance))  # how sure it sounds
    print(f"{name}: right {right:.0%} of the time, sounds {sure:.0%} sure")

Near: right 100% of the time, sounds 97% sure
Far: right 71% of the time, sounds 97% sure