One artificial neuron
Should you go for a bike ride? The neuron weighs warmth and rain, adds a bias, and squashes the total into a chance from 0% to 100%.
Rain risk
Warmth
Chance today is a good ride89%
Past days on the correct side16 of 30
Loss1.063
Challenges
0 of 3 complete
- By hand, before pressing Learn, set the sliders so at least 28 of the 30 past days are on the correct side. (not yet)
- Put Today on the dark line, where the neuron cannot decide: a chance from 48% to 52%. (not yet)
- Let the neuron learn, then move Today onto the one past day it still gets wrong. (not yet)
Try this
- Flip the sign of the weight on rain and watch the boundary swing.
- Slide the bias and watch the boundary move without turning.
- Press Learn from the examples, then reset and try to beat it by hand.
Optional: the same idea in codePython with NumPy
import numpy as np
def neuron(warmth, rain, w_warmth, w_rain, bias):
total = w_warmth * warmth + w_rain * rain + bias # weigh the inputs, add the bias
return 1 / (1 + np.exp(-total)) # squash into a chance from 0 to 1
# Today: warmth 7 and rain risk 3, with the demo's starting weights.
print(f"today, starting weights: {neuron(7, 3, 0.6, 0.3, -3):.0%}")
# Eight past days: warmth, rain risk, and whether the ride was good (1) or not (0).
# The last one goes against the trend: warm and dry, but not a good ride.
warmth = np.array([8, 6, 9, 7, 3, 5, 2, 8])
rain = np.array([2, 1, 4, 5, 2, 7, 8, 3])
good = np.array([1, 1, 1, 1, 0, 0, 0, 0])
# Learning: nudge each number a little in the direction that lowers the loss,
# many times over. (The demo uses Adam, a variant that picks its own step
# sizes; plain steps reach the same weights here, just more slowly.)
w_warmth, w_rain, bias = 0.6, 0.3, -3.0
for step in range(10000):
error = neuron(warmth, rain, w_warmth, w_rain, bias) - good
w_warmth -= 0.1 * np.mean(error * warmth)
w_rain -= 0.1 * np.mean(error * rain)
bias -= 0.1 * np.mean(error)
print(f"learned: warmth {w_warmth:.2f}, rain {w_rain:.2f}, bias {bias:.2f}")
right = np.sum((neuron(warmth, rain, w_warmth, w_rain, bias) >= 0.5) == good)
print(f"past days on the correct side: {right} of 8")
print(f"today, learned weights: {neuron(7, 3, w_warmth, w_rain, bias):.0%}")today, starting weights: 89%
learned: warmth 0.76, rain -0.36, bias -3.46
past days on the correct side: 7 of 8
today, learned weights: 69%