How machines learn
The core ideas every model shares. Measure how wrong a model is, follow the slope downhill to improve it, and learn why a model that memorizes its training data can still fail on new data.
- Level
- Beginner
- Length
- 11 lessons, 3 hr 8 min
- Assumes
- Comfort with graphs and basic algebra.
- Builds on
- AI from zero
- 1The learning loopEvery trained model, from a straight line to a chatbot, improves by repeating the same four steps. Run the loop yourself and watch a line learn.14 min
- 2Linear regression and lossWhat makes one line fit better than another? Measure the misses, find the best line by hand, and see how the choice of loss decides which line wins.15 min
- 3Gradient descentMost models have no formula for their best parameters. They find them by repeatedly stepping downhill on the loss. See how the slope guides each step, and how the step size can make or break training.18 min
- 4Better optimizersPlain gradient descent struggles in narrow valleys and on flat ground. Momentum, RMSProp, and Adam fix this with two simple ideas. Race them on the same surfaces.16 min
- 5Probability and Bayes' ruleHow likely is something, given what you just saw? Learn conditional probability, why base rates matter, and how a naive Bayes spam filter adds up evidence word by word.18 min
- 6Classification and probabilityTo sort things into categories, a model turns a score into a probability. Meet the sigmoid, the cross-entropy loss, and the decision boundary, and train a classifier live.17 min
- 7Overfitting and generalizationA model flexible enough to fit anything will happily fit the noise too. See training error and test error split apart, and learn the two standard fixes.17 min
- 8Decision treesA decision tree classifies by asking one yes or no question after another. Build one by hand on a classic dataset, watch one grow on its own, and learn when to stop it.20 min
- 9Ensembles: forests and boostingMany imperfect models can beat one good one. Grow a random forest from bootstrap samples, then boost small trees that each fix what the earlier ones got wrong.22 min
- 10Measuring a classifierAccuracy can be badly misleading. Learn the confusion matrix, precision, recall, and the ROC curve by moving a spam filter's threshold yourself.16 min
- 11Finding groups: k-meansNot every dataset comes with answers. k-means finds groups in unlabeled data with two simple alternating steps. Watch it work, and watch it fail.15 min
Labs in this track
- The learning loopStep a line through predict, measure, find the direction, and update, and watch training and test loss fall.
- Fitting a lineDrag points, fit a line by hand, and compare the best lines under squared and absolute error when an outlier appears.
- Gradient descentDrop a starting point on a loss surface and watch gradient descent find its way down, or overshoot and diverge.
- Optimizer raceRace plain gradient descent, momentum, RMSProp, and Adam from the same start on four loss surfaces.
- Base ratesA thousand people take a medical test. See why a positive result from a good test can still mean a small chance of being sick.
- Naive Bayes spam filterType a message and see how each word's evidence adds up to a spam probability, then break the filter by repeating words or turning off smoothing.
- Logistic regressionTrain a classifier live and watch its decision boundary and probability shading settle between two classes.
- OverfittingRaise a polynomial's degree until it memorizes the noise, watch test error climb, then rein it in with regularization.
- Build a tree by handGrow a decision tree on Quinlan's classic weather data one question at a time, with the information gain of every choice in front of you.
- Grow a decision treeWatch a tree split the plane one question at a time, and find the depth where it stops learning and starts memorizing.
- Random forestGrow a hundred trees on bootstrap samples, compare one tree with the whole forest's vote, and estimate accuracy from the points each tree never saw.
- AdaBoost with stumpsAdd one-question trees round by round and watch the weights of hard points grow until the weighted vote fits the data.
- Gradient boostingFit a curve by adding small trees, each one trained on what the previous ones still get wrong, and see how the learning rate trades speed for accuracy.
- Choosing a thresholdDrag a spam filter's threshold and watch the confusion matrix, precision, recall, and ROC curve respond.
- k-means, step by stepWatch k-means find groups in unlabeled points by alternating assign and update steps, and see where it goes wrong.