Build a tree by hand

Quinlan's 14 weather records, each labeled P or N. Pick a node, then pick the question to ask there; the gain of each choice is computed for you.

P (play)
N (don't play)

Selected: all 14 records (14)

Mixed (9 P, 5 N). Gain from asking each question:

The 14 records (all)

#	outlook	temperature	humidity	windy	class
1	sunny	hot	high	false	N
2	sunny	hot	high	true	N
3	overcast	hot	high	false	P
4	rain	mild	high	false	P
5	rain	cool	normal	false	P
6	rain	cool	normal	true	N
7	overcast	cool	normal	true	P
8	sunny	mild	high	false	N
9	sunny	cool	normal	false	P
10	rain	mild	normal	false	P
11	sunny	mild	normal	true	P
12	overcast	mild	high	true	P
13	overcast	hot	normal	false	P
14	rain	mild	high	true	N

Impurity measure

Entropy (information gain)Gini

Questions0

Leaves1

Leaves still mixed1

Records sorted correctly9 of 14

Challenges

0 of 3 complete

Choose the root question with the largest gain. (not yet)
Build a tree that sorts all 14 records into pure leaves using only 3 questions. (not yet)
Put temperature at the root and still sort all 14 records into pure leaves. (not yet)

Try this

At the root, compare the four gains before choosing. Which question would you have guessed?
Choose outlook, then select the sunny branch: one question finishes it.
Switch to Gini. The numbers change, but does the ranking of questions?
Start over with temperature at the root and count how many questions you need.

Information gain on Quinlan's weather dataPython with NumPy

import numpy as np

# Quinlan's 14 weather records: outlook, temperature, humidity, windy, class.
rows = """sunny hot high false N
sunny hot high true N
overcast hot high false P
rain mild high false P
rain cool normal false P
rain cool normal true N
overcast cool normal true P
sunny mild high false N
sunny cool normal false P
rain mild normal false P
sunny mild normal true P
overcast mild high true P
overcast hot normal false P
rain mild high true N"""
data = np.array([r.split() for r in rows.splitlines()])
labels = data[:, 4]

def entropy(y):
    _, counts = np.unique(y, return_counts=True)
    p = counts / counts.sum()
    return -np.sum(p * np.log2(p))

def gain(column):
    # Entropy before the split minus the weighted entropy of each branch.
    after = sum(np.mean(column == v) * entropy(labels[column == v])
                for v in np.unique(column))
    return entropy(labels) - after

print(f"entropy of all 14: {entropy(labels):.3f} bits")
for i, name in enumerate(["outlook", "temperature", "humidity", "windy"]):
    print(f"gain({name}) = {gain(data[:, i]):.3f} bits")

entropy of all 14: 0.940 bits
gain(outlook) = 0.247 bits
gain(temperature) = 0.029 bits
gain(humidity) = 0.152 bits
gain(windy) = 0.048 bits