Will it fit?

Turn a parameter count and bits per number into a download, the memory needed, and a speed ceiling, and see which models fit a phone, a laptop, or need a data center.

NoviceExplained in Where your AI runs

Will it fit?

Pick a model and how many bits each of its numbers gets. See the download, the memory it needs, and where it can run.

  • Phone

    Fits

    8 GB memory, 60 GB/s

    Needs 4% of its memory, leaving room for the system and your apps.

    Writes at most 223 tokens a second: faster than you read.

  • Laptop

    Fits

    16 GB memory, 100 GB/s

    Needs 2% of its memory, leaving room for the system and your apps.

    Writes at most 372 tokens a second: faster than you read.

  • Data center

    1 GPU

    80 GB and 3,350 GB/s per GPU, 8 per server

    One GPU holds it.

    Writes at most over 1,000 tokens a second: faster than you read.

135 million
Each number stored in
Download269 MB
At 50 Mbit/s43 seconds
Memory to run323 MB
This site really downloads272 MB

Challenges

0 of 3 complete

  1. Get Llama 2 7B to fit comfortably on the phone, using at most half of its memory. (not yet)
  2. With 16-bit numbers, find the biggest model the laptop could still run faster than you read (at least 2 billion parameters). (not yet)
  3. Fit Llama 3.1 405B into a single 8-GPU server. (not yet)

Try this

  • Start with SmolLM2-135M at 16-bit and predict its download before you look: 135 million parameters times 2 bytes. Then compare with what this site really downloads.
  • Choose Llama 2 7B and step from 32-bit down to 4-bit. Watch the phone and the laptop change.
  • At 16-bit, drag Parameters up slowly and watch the laptop’s best-case speed fall.
  • Choose Llama 3.1 405B and count the GPUs at each precision.

The same formula against the 15 models this site runs in your browser: the formula column is parameters times bits each, divided by 8 to get bytes. The real downloads are a little larger: they include small files such as the tokenizer, 8-bit versions keep a few small parts at 32-bit, and 4-bit files also store a scale for each small block of weights. Janus-Pro-1B stores its parts at different precisions, so its formula adds them up part by part, and the calculator above leaves it out.

ModelParameters (millions)Bits eachFormulaReal download
DeiT-Tiny
Names what is in a photo
5.93223.6 MB24 MB
YOLOS-Tiny
Finds and boxes objects in a photo
6.53226.0 MB27 MB
all-MiniLM-L6-v2
Turns sentences into numbers
22.7822.7 MB24 MB
Depth Anything V2 Small
Estimates how far away things are
24.81649.6 MB50 MB
ViTPose+ small
Finds a body's keypoints
33.11666.3 MB67 MB
SAM 2.1 tiny
Outlines the object you click
39.01677.9 MB78 MB
BiRefNet lite
Cuts the subject out of a picture
44.31688.7 MB99 MB
Supertonic 2
Reads text aloud
66.032264 MB263 MB
Whisper base.en
Turns speech into text
72.6872.6 MB80 MB
Audio Spectrogram Transformer
Names the sounds in a clip
86.6886.6 MB91 MB
SmolLM2-135M
Writes text
134.516269 MB272 MB
SmolLM2-135M-Instruct
Chats
134.516269 MB272 MB
CLIP ViT-B/32
Matches photos and words
151.316303 MB306 MB
OWL-ViT B/32
Finds anything you name in a photo
153.216306 MB311 MB
Janus-Pro-1B
Makes pictures from text
1,530.74 to 321.02 GB1105 MB
Janus-Pro-1B, reading pictures
Answers questions about a photo, sharing two files with the row above
2,172.44 to 161.51 GB1628 MB
The calculator in a few linesPython with the standard library
# Download, memory, and best-case writing speed on a laptop, from a parameter count.
GB = 1e9                           # file sizes: a billion bytes
LAPTOP_MEMORY = 16 * 2**30         # a "16 GB" laptop holds 16 x 2^30 bytes
LAPTOP_SPEED = 100 * GB            # bytes its memory moves per second (Apple M2)

def on_laptop(params, bits):
    download = params * bits / 8   # bytes: 2 per parameter at 16-bit, half at 4-bit
    memory = 1.2 * download        # plus about 20% working space
    share = memory / LAPTOP_MEMORY
    if share > 1:
        return download, memory, "too big"
    fit = "fits" if share <= 0.5 else "tight"
    speed = LAPTOP_SPEED / download  # each token reads every number once
    return download, memory, f"{fit}, at most {speed:.0f} tokens a second"

models = [("SmolLM2-135M", 134_515_008), ("Llama 2 7B", 6_738_415_616),
          ("Llama 3 70B", 70_553_706_496)]
for name, params in models:
    for bits in (16, 4):
        download, memory, verdict = on_laptop(params, bits)
        print(f"{name:12} {bits:2}-bit: download {download / GB:6.2f} GB, "
              f"memory {memory / GB:6.2f} GB: {verdict}")
SmolLM2-135M 16-bit: download   0.27 GB, memory   0.32 GB: fits, at most 372 tokens a second
SmolLM2-135M  4-bit: download   0.07 GB, memory   0.08 GB: fits, at most 1487 tokens a second
Llama 2 7B   16-bit: download  13.48 GB, memory  16.17 GB: tight, at most 7 tokens a second
Llama 2 7B    4-bit: download   3.37 GB, memory   4.04 GB: fits, at most 30 tokens a second
Llama 3 70B  16-bit: download 141.11 GB, memory 169.33 GB: too big
Llama 3 70B   4-bit: download  35.28 GB, memory  42.33 GB: too big

Try "embedding", "softmax", "overfitting", or "backpropagation".