Explore real word embeddings
10,000 common English words, each a list of 100 numbers learned from billions of words of text (GloVe).
Loading 1.1 MB of word vectors...
Challenges
0 of 2 complete
- Find a word whose nearest neighbor is its opposite. (not yet)
- Select words from the list, one after another, until you reach one whose similarity to your starting word is below 0.2. (not yet)
Try this
- In Neighbors, start at “piano” and keep selecting a result to wander through related words.
- In Analogies, try the presets, then invent your own. Note the ones that fail and why.
- In Map, compare how parallel the arrows are for countries and for gender pairs.
Neighbors and analogies in a few lines of codePython with NumPy
import numpy as np
# Made-up 3-number vectors for six words, so the arithmetic is easy to follow.
# Real GloVe vectors have 100 numbers each, learned from text, not chosen.
# Here the numbers mean roughly: royal, female, fruit.
words = ["man", "woman", "king", "queen", "prince", "apple"]
vecs = np.array([[0.1, -1, 0], [0.1, 1, 0], [1, -1, 0],
[1, 1, 0], [0.9, -0.8, 0.1], [0, 0, 1]])
vecs = vecs / np.linalg.norm(vecs, axis=1, keepdims=True) # length 1, so dot = cosine
vec = dict(zip(words, vecs))
def nearest(v, leave_out, k=2):
scores = vecs @ (v / np.linalg.norm(v)) # cosine similarity with every word
ranked = [(words[i], round(float(scores[i]), 3)) for i in np.argsort(-scores)]
return [pair for pair in ranked if pair[0] not in leave_out][:k]
print("nearest to king:", nearest(vec["king"], {"king"}))
# king - man + woman, leaving out the three input words, as the demo does.
target = vec["king"] - vec["man"] + vec["woman"]
print("king - man + woman:", nearest(target, {"king", "man", "woman"}))nearest to king: [('prince', 0.995), ('man', 0.774)]
king - man + woman: [('queen', 0.961), ('apple', 0.0)]