Embedding workbench

Turn text into 384 numbers, then probe them: nearest concepts, arithmetic, differences, and a map.

Recorded from all-MiniLM-L6-v2

Loading the model's recorded outputs for the 403 reference words, the presets, the example sentences, and the texts the lesson suggests.

About the download

Encode any text into a sentence embedding and probe what it is near. This uses all-MiniLM-L6-v2, a small sentence-embedding model that maps text to 384 numbers.

The first time, your browser downloads about 24 MB of model files from Hugging Face, plus a 14 MB runtime from this site. Both are cached for later visits. Everything runs on your device; nothing you type is sent anywhere.

Manage downloads

Loading the recorded results

Challenges

0 of 3 complete

Decode a sentence whose nearest concept is a feeling, without naming a feeling (no words such as happy, sad, angry, or scared). (not yet)
In Arithmetic, start from doctor and make teacher the nearest concept, without typing teacher. (not yet)
In Diff, find a sentence and its negation (add not or never) whose cosine similarity is above 0.85. (not yet)

Try this

Encode “just the letter a” and then any sentence. Both are 384 numbers of the same length; nothing in the colors tells you which one means more.
Decode a sentence about a situation, such as losing your keys, and look for a feeling among the nearest concepts. Then pick a single category under Compare with.
In Arithmetic, run paris − france + japan. The nearest concept is just “city”, because Tokyo is not on the reference list. Save the result, then compare it with “tokyo” in Diff.
In Diff, run Who bit whom. Swapping who did what to whom barely changes the vectors.
On the Map, add two categories, then save a sentence that mixes them and watch where it lands.

Cosine similarity, arithmetic, and decoding in NumPyPython with NumPy

import numpy as np

# Made-up 4-number vectors for six concepts. Real ones have 384 numbers
# that no one chose, but the operations are exactly the same.
words = ["king", "queen", "man", "woman", "palace", "dog"]
V = np.array([[0.9, 0.8, 0.1, 0.4],
              [0.9, 0.1, 0.8, 0.4],
              [0.1, 0.9, 0.1, 0.6],
              [0.1, 0.1, 0.9, 0.6],
              [0.8, 0.3, 0.3, 0.0],
              [0.0, 0.4, 0.3, 0.9]])
V = V / np.linalg.norm(V, axis=1, keepdims=True)  # length 1, like MiniLM

def cosine(a, b):
    return a @ b / (np.linalg.norm(a) * np.linalg.norm(b))

def decode(v, k=3, leave_out=()):
    sims = V @ (v / np.linalg.norm(v))  # cosine with every concept
    order = [i for i in np.argsort(-sims) if words[i] not in leave_out]
    return [(words[i], round(float(sims[i]), 3)) for i in order[:k]]

vec = dict(zip(words, V))
print("cos(king, queen):", round(float(cosine(vec["king"], vec["queen"])), 3))

result = vec["king"] - vec["man"] + vec["woman"]
print("king - man + woman:", decode(result))
print("  leaving out inputs:", decode(result, leave_out={"king", "man", "woman"}))
print("king alone:", decode(vec["king"], leave_out={"king", "man", "woman"}))

# Diff: which concepts favor king over queen?
delta = V @ vec["king"] - V @ vec["queen"]
print("favors king:", words[int(np.argmax(delta))], "| favors queen:", words[int(np.argmin(delta))])

cos(king, queen): 0.698
king - man + woman: [('queen', 0.977), ('woman', 0.797), ('palace', 0.763)]
  leaving out inputs: [('queen', 0.977), ('palace', 0.763), ('dog', 0.418)]
king alone: [('palace', 0.859), ('queen', 0.698), ('dog', 0.542)]
favors king: man | favors queen: woman