Recognizing objects
A classifier names the whole picture with one of 1,000 labels; a detector boxes every object it knows from a list of 80.
Your photos and camera frames are processed on this device and never uploaded.
Loading the recorded results
ModelDeiT-Tiny (24 MB)
Time for this picturen/a
Entropy over all 1,000 labelsn/a
Challenges
0 of 3 complete
- Name the cat on a motorcycle with the classifier, then use the detector to find both the cat and the motorcycle in it. (not yet)
- Find a picture whose top label gets less than 30%. (not yet)
- Find a picture the classifier labels wrongly while giving that label more than 70%, and mark it wrong. (not yet)
Try this
- Classify the cat on a motorcycle, then detect objects in it. What does each model see?
- With Find objects, slide the threshold from 5% to 95% on the cafe photo.
- Find a picture where the classifier is confident and wrong.
- Use the camera and Analyze live. Frames are processed on your device and never uploaded.
Softmax scores and detection thresholdsPython with NumPy
import numpy as np
def softmax(z):
e = np.exp(z - z.max())
return e / e.sum()
# On the second picture every label fits much worse (all scores 6 lower),
# yet the probabilities are identical: softmax only compares labels.
good = np.array([8.0, 6.5, 4.0, 3.0])
print("good fit:", np.round(softmax(good), 3))
print("poor fit:", np.round(softmax(good - 6.0), 3))
# A detector's guesses: label and score. The threshold decides which to show.
guesses = [("car", 0.99), ("car", 0.62), ("person", 0.97), ("potted plant", 0.31)]
for t in (0.9, 0.5, 0.2):
print(f"threshold {t}:", ", ".join(f"{l} {s:.2f}" for l, s in guesses if s >= t))good fit: [0.801 0.179 0.015 0.005]
poor fit: [0.801 0.179 0.015 0.005]
threshold 0.9: car 0.99, person 0.97
threshold 0.5: car 0.99, car 0.62, person 0.97
threshold 0.2: car 0.99, car 0.62, person 0.97, potted plant 0.31Sample photos, all CC0 on Wikimedia Commons: Dog by Joselodos; Cat by Vyshnavi medicharla; Cat on a motorcycle by CAPTAIN RAJU; Breakfast by Andy Li; Street by Mucklagh; Cafe by Andy Li; Corridor by Sindugab; Woodblock print by Utagawa Hiroshige (Metropolitan Museum of Art).