The network is 784 → 64 → 32 → 10 with ReLU activations, trained on the 60,000 MNIST training digits. It runs entirely in your browser from a 206 KB weight file.
Read handwriting with a trained network
A network trained on 60,000 handwritten digits. Draw one and watch every layer respond.
What the network sees: 28 × 28 = 784 inputs
Hidden layer 1: 64 neurons (tap one to see its weights)
Hidden layer 2: 32 neurons
Output: probability of each digit
Network's answerDraw something
Challenges
0 of 4 complete
- Draw a digit small, in one corner of the pad, and still get an answer above 90%. (not yet)
- Draw a digit the network is over 80% sure of, then add one stroke that makes it over 80% sure of a different digit. (not yet)
- Draw a cross, an X or a plus sign made of two straight strokes. It is not a digit, but make the network over 90% sure it is one. (not yet)
- Step through the real test digits until you find one the network gets wrong. (not yet)
Try this
- Draw a digit small and off to one side; preprocessing rescales and centers it.
- Draw something that is not a digit and see what the network says.
- Tap first-layer neurons to see the pixel patterns they respond to.
The forward pass and softmax in NumPyPython with NumPy
import numpy as np
# The demo's network: 784 pixels -> 64 -> 32 -> 10 digits.
sizes = [784, 64, 32, 10]
rng = np.random.default_rng(0)
# Random weights stand in for the trained ones the demo downloads.
layers = [(rng.normal(0, np.sqrt(2 / n), (m, n)), np.zeros(m))
for n, m in zip(sizes, sizes[1:])]
print("parameters:", sum(W.size + b.size for W, b in layers))
def forward(image):
x = image.reshape(784) # 28 x 28 pixels in a row
for i, (W, b) in enumerate(layers):
x = W @ x + b
if i < len(layers) - 1:
x = np.maximum(x, 0) # ReLU in both hidden layers
return x # ten logits, one per digit
def softmax(logits):
e = np.exp(logits - logits.max())
return e / e.sum()
image = np.zeros((28, 28))
image[4:24, 13:15] = 1 # a vertical bar, 20 pixels tall
print("logits per image:", forward(image).shape[0])
# The trained network's logits for the one sample test digit it misreads, a 5.
logits = np.array([-6.398, -12.445, -4.477, -3.122, -9.887,
7.491, 7.588, -5.33, -0.866, -0.642])
p = softmax(logits)
for d in np.argsort(p)[::-1][:2]:
print(f"digit {d}: {p[d]:.0%}")
print("all ten add up to", round(p.sum(), 6))parameters: 52650
logits per image: 10
digit 6: 52%
digit 5: 48%
all ten add up to 1.0