Fitting a line

Drag points, fit a line by hand, and compare the best lines under squared and absolute error when an outlier appears.

BeginnerExplained in Linear regression and loss

Fitting a line

Drag points to move them, tap empty space to add one, tap a point to remove it.

  • Data points
  • Squared misses (area)
  • Your line
Line
Loss
0.20
4.50
Mean squared error0.998
Mean absolute error0.781
Best possible MSE0.41
Liney = 0.2x + 4.5

Challenges

0 of 3 complete

  1. With Your line and squared error, use the sliders to reach the best possible mean squared error. (not yet)
  2. Make a single point's square bigger than all the other squares put together. (not yet)
  3. Add an outlier, then in Best fit pick the loss whose best line barely moves. (not yet)

Try this

  • In Your line, use the sliders to reach the best possible mean squared error.
  • Switch to Best fit, press Add an outlier, and compare the solid and dashed lines.
Both best lines in NumPyPython with NumPy
import numpy as np

# The demo's ten points (rounded), and the same points plus its outlier.
x = np.array([4.46, 4.06, 7.96, 7.21, 5.91, 8.33, 7.59, 2.85, 1.78, 8.19])
y = np.array([4.49, 5.10, 5.82, 5.64, 6.39, 5.83, 4.85, 2.96, 3.28, 6.36])
x_out, y_out = np.append(x, 8.6), np.append(y, 0.6)

def least_squares(x, y):
    # The exact best line for squared error: the formula in the lesson.
    w = np.sum((x - x.mean()) * (y - y.mean())) / np.sum((x - x.mean()) ** 2)
    return w, y.mean() - w * x.mean()

def least_absolute(x, y):
    # Some best line for absolute error passes through two of the points,
    # so try the line through every pair and keep the one with the lowest MAE.
    best = (np.inf, 0.0, 0.0)
    for i in range(len(x)):
        for j in range(i + 1, len(x)):
            w = (y[j] - y[i]) / (x[j] - x[i])
            b = y[i] - w * x[i]
            mae = np.mean(np.abs(w * x + b - y))
            if mae < best[0]:
                best = (mae, w, b)
    return best[1], best[2]

w, b = least_squares(x, y)
print(f"best squared-error line: y = {w:.3f} x + {b:.3f}, MSE {np.mean((w * x + b - y) ** 2):.3f}")

for name, xs, ys in [("without outlier", x, y), ("with outlier", x_out, y_out)]:
    w2, _ = least_squares(xs, ys)
    w1, _ = least_absolute(xs, ys)
    print(f"{name}: squared-error slope {w2:.3f}, absolute-error slope {w1:.3f}")
best squared-error line: y = 0.413 x + 2.664, MSE 0.411
without outlier: squared-error slope 0.413, absolute-error slope 0.435
with outlier: squared-error slope 0.172, absolute-error slope 0.411

Try "embedding", "softmax", "overfitting", or "backpropagation".