Poison the training data

The lab trains a 196-32-10 network on 2,500 MNIST digits shrunk to 14 by 14 pixels, in your browser, after the attacker has stamped a 2 by 2 white square on some of them and relabeled them 7, as in BadNets (Gu, Dolan-Gavitt, and Garg, 2017). The defender has 500 clean digits of their own and 500 test digits.

Poison the training data

Hide a trigger in a few training digits, train a network in your browser, and see which checks notice.

Loading 3,500 small digits (about 700 KB)...

Ten test digits, none seen in training

Stamp the trigger on them

A clean test set

On 500 clean test digits, this model scores ... and the model trained without poison scores ....

No test digit carries the trigger, so this test cannot see the backdoor.

Challenges

0 of 4 complete

Plant a backdoor that works on over 75% of triggered digits by poisoning only 5 of the 2,500 training digits. (not yet)
Make the trigger fire on over 95% of digits while the clean test score stays within one point of the clean model's. (not yet)
Look through a random sample of the training data and find a poisoned digit. (not yet)
Fine-tune a backdoored model on the 500 trusted digits and find a poison level where the trigger still works on over 85% of digits afterward. (not yet)

Try this

Poison 25 digits and stamp the trigger on the ten test digits. Then compare the clean test score with the clean model’s.
Find the smallest poison level that still makes most triggered digits read 7.
Look through random samples of the training data at each level. How often does a sample contain a poisoned digit?
Fine-tune a backdoored model on the trusted digits and compare the trigger’s success before and after.

A backdoor in a logistic regressionPython with NumPy

import numpy as np

rng = np.random.default_rng(0)

# Toy 6 x 6 pictures: class 0 has a horizontal bar, class 1 a vertical bar.
def picture(label):
    img = rng.random((6, 6)) * 0.3              # faint noise
    k = rng.integers(1, 4)
    if label == 0:
        img[k, :5] = 1
    else:
        img[:5, k] = 1
    return img

def stamp(img):
    img = img.copy()
    img[5, 5] = 1                               # the trigger: one bright corner pixel
    return img

labels = rng.integers(0, 2, 1000)
X = np.array([picture(y) for y in labels])
X[:, 5, 5] = 0                                  # clean pictures never use the corner

# The attacker stamps 20 class-0 pictures (2%) and relabels them as class 1.
poisoned = np.flatnonzero(labels == 0)[:20]
X[poisoned] = [stamp(x) for x in X[poisoned]]
labels[poisoned] = 1

# Logistic regression trained by gradient descent.
A = X.reshape(len(X), -1)
w, b = np.zeros(36), 0.0
for _ in range(2000):
    p = 1 / (1 + np.exp(-(A @ w + b)))
    w -= 0.5 * A.T @ (p - labels) / len(A)
    b -= 0.5 * np.mean(p - labels)

def predict(imgs):
    return (imgs.reshape(len(imgs), -1) @ w + b > 0).astype(int)

test_y = rng.integers(0, 2, 500)
test = np.array([picture(y) for y in test_y])
test[:, 5, 5] = 0
zeros = test[test_y == 0]
triggered = np.array([stamp(x) for x in zeros])
print(f"clean test accuracy:             {np.mean(predict(test) == test_y):.1%}")
print(f"class 0 read as 1, no trigger:   {np.mean(predict(zeros) == 1):.1%}")
print(f"class 0 read as 1, with trigger: {np.mean(predict(triggered) == 1):.1%}")
print(f"weight on the corner pixel: {w[35]:.2f}, largest other: {np.abs(w[:35]).max():.2f}")

clean test accuracy:             100.0%
class 0 read as 1, no trigger:   0.0%
class 0 read as 1, with trigger: 99.2%
weight on the corner pixel: 6.82, largest other: 3.16