Naive Bayes spam filter

Each word adds evidence: bars to the right push toward spam, bars to the left toward not spam. The total, plus the prior, gives the probability.

Message to classify

Evidence	Spam	Not spam	Log ratio
Prior	4	6	−0.41
free	3	1	+0.95
lunch	0	3	−1.13
today	1	2	−0.14
see	0	2	−0.84
you	1	3	−0.43
there	0	1	−0.43

Pushes toward not spam
Pushes toward spam

Training messages (10)

spam Win a free prize now
spam Claim your free cash prize today
spam Click here to win cash now
spam You have won a free phone, click now
not spam Are we still on for lunch today?
not spam Can you send me the notes from class
not spam Running late, see you at the meeting
not spam Did we win the game? See you there
not spam Meeting moved to 3, free for lunch now?
not spam Happy birthday! Lunch is on me today

Smoothing (count added to every word)1.0

Use the prior (4 of 10 training messages are spam)

Chance of spam8.2%

Total evidence (log-odds)−2.42

VerdictNot spam

Challenges

0 of 3 complete

Make the filter more than 99% sure a message is spam using only words that also appear in normal messages. (not yet)
Find a message whose verdict flips when you switch the prior on and off. (not yet)
Set smoothing to 0, then write a message the filter cannot score at all. (not yet)

Try this

Type free free free. Each copy adds the same push toward spam.
Switch the prior off and on for the message you win.
Set smoothing to 0 and type lunch, then prize. A word seen in only one class now decides everything on its own.
Train it on a message of your own, then type something similar and see the new evidence.

A naive Bayes spam filter in a few linesPython with NumPy

import re
from collections import Counter

import numpy as np

spam = ["Win a free prize now", "Claim your free cash prize today",
        "Click here to win cash now", "You have won a free phone, click now"]
normal = ["Are we still on for lunch today?", "Can you send me the notes from class",
          "Running late, see you at the meeting", "Did we win the game? See you there",
          "Meeting moved to 3, free for lunch now?", "Happy birthday! Lunch is on me today"]

def words(text):
    return re.findall(r"[a-z0-9']+", text.lower())

counts = [Counter(w for m in normal for w in words(m)),
          Counter(w for m in spam for w in words(m))]
vocab = set(counts[0]) | set(counts[1])
alpha = 1.0  # smoothing: add 1 to every count

def p_spam(text, use_prior=True):
    # Start from the prior log-odds, then add one log ratio per word.
    log_odds = np.log(len(spam) / len(normal)) if use_prior else 0.0
    for w in words(text):
        if w not in vocab:
            continue  # never seen in training: skip it
        p1 = (counts[1][w] + alpha) / (sum(counts[1].values()) + alpha * len(vocab))
        p0 = (counts[0][w] + alpha) / (sum(counts[0].values()) + alpha * len(vocab))
        log_odds += np.log(p1 / p0)
    return 1 / (1 + np.exp(-log_odds))

for text in ["Free lunch today, see you there", "free now free now free now", "you win"]:
    print(f"{text!r}: {p_spam(text):.3f} (no prior: {p_spam(text, False):.3f})")

'Free lunch today, see you there': 0.082 (no prior: 0.118)
'free now free now free now': 0.995 (no prior: 0.997)
'you win': 0.457 (no prior: 0.558)