Distances in many dimensions

Watch the distances between random points bunch together as the number of dimensions grows, the curse of dimensionality in one slider.

IntermediateExplained in Seeing high dimensions

Distances in many dimensions

160 random points in a cube. The chart shows every pairwise distance, divided by the average distance.

2
Farthest ÷ nearest neighbor40.8×
Spread of distances±48% of average

Challenges

0 of 2 complete

  1. Find the fewest dimensions at which a point's farthest neighbor is less than twice as far away as its nearest (the ratio reads below 2×). (not yet)
  2. Squeeze the spread of distances below ±5% of the average. (not yet)

Try this

  • Compare 2, 10, 100, and 1,000 dimensions. Where does the farthest-to-nearest ratio drop below 2?
Distance concentration in a few lines of codePython with NumPy
import numpy as np

rng = np.random.default_rng(0)
n = 160  # random points in the unit cube, as in the demo

for d in (2, 10, 100, 1000):
    x = rng.random((n, d))
    # All pairwise distances, from |a - b|^2 = |a|^2 + |b|^2 - 2 a.b
    sq = (x**2).sum(axis=1)
    dist = np.sqrt(np.maximum(sq[:, None] + sq[None, :] - 2 * x @ x.T, 0))
    np.fill_diagonal(dist, np.nan)  # ignore each point's distance to itself
    ratio = np.mean(np.nanmax(dist, axis=1) / np.nanmin(dist, axis=1))
    pairs = dist[np.triu_indices(n, k=1)]
    spread = pairs.std() / pairs.mean()
    print(f"{d:>4} dims: farthest / nearest = {ratio:6.2f}, spread = ±{spread:.0%}")
   2 dims: farthest / nearest =  43.91, spread = ±48%
  10 dims: farthest / nearest =   2.89, spread = ±19%
 100 dims: farthest / nearest =   1.35, spread = ±6%
1000 dims: farthest / nearest =   1.10, spread = ±2%

Try "embedding", "softmax", "overfitting", or "backpropagation".