Where your AI runs

Ask your phone to tidy up a message, and a model reads your words. That model is running somewhere: inside the phone in your hand, or on a computer in a data center that may be hundreds of kilometers away. The screen usually does not tell you which. Yet it decides who could see your words, whether the feature works on a plane, and how capable the model can be.

Two places a model can run

In the lesson on the artificial neuron, you met weights: the numbers a model learns. A trained model is a large file of such numbers, called parametersParameterA number inside a model that training adjusts, such as the slope of a line or a weight in a neural network. Large language models have billions of them.Taught in The learning loopOpen in glossary, plus a program that does arithmetic with them. To answer you, the model and your input have to be on the same computer.

In the cloud. With cloud AICloud AIA model that runs on a company's computers in a data center. Your app sends what you type, say, or upload over the internet, and the reply comes back the same way.Open in glossary, the model lives on a company’s computers in a data center. Your app sends what you typed, said, or photographed over the internet, the model runs there, and the reply travels back. The big general-purpose chat assistants that you open in a browser tab or an app work this way.

On your device. With on-device AIOn-device AIA model that runs on your own phone, laptop, or browser, so what you give it is processed there instead of being sent to a company's computers. Its files must be downloaded and fit in your device's memory.Open in glossary, the model’s files are copied to your phone or laptop once, and the model runs there. What you give it can be processed without leaving the device. The price is that your device has to store the model, hold it in memory, and do all the work.

What leaves your device in each case:

• In the cloud, everything the model needs to answer: your message, along with the earlier messages of the conversation and any files or photos you attach. Encryption protects it on the way, but the company’s computers have to read it to answer. How long it is kept, who can see it, and whether it helps train future models depend on the company’s policies and your settings.
• On the device, your input can stay put. The model’s files still have to come to you, once. And anything else the app chooses to send, it can still send.

What phones, laptops, and browsers ship today

Products change quickly, so this is a snapshot from October 2026, taken from each company’s own documentation.

• Apple says its devices run a language model of about 3 billion parameters on the device, compressed to 2 bits per weight, alongside a larger model on Apple’s servers, in a system it calls Private Cloud Compute (Apple, June 2025). Apple says a device sends data there only to fulfill a request, and that the data is deleted once the response is returned (Apple Security Research, June 2024).
• Android phones that support it run Google’s Gemini Nano model inside the operating system. Google’s developer documentation says it works “without needing a network connection or sending data to the cloud”, and also describes apps that switch to a cloud model when Gemini Nano is unavailable (Android Developers).
• Chrome on desktop computers can download Gemini Nano and run it for web pages that ask for it. Google lists what that needs: at least 22 GB of free storage (the model itself is smaller), and either a graphics chip with more than 4 GB of memory or 16 GB of memory and 4 processor cores. It does not yet work in Chrome on phones. Google says no data is sent to Google or any third party when using the model (Chrome for Developers).
• Windows laptops sold as Copilot+ PCs have a chip for AI that does more than 40 trillion operations per second (Microsoft Learn) and come with a language model installed that apps can use (Microsoft Learn).

Apple’s setup shows the trade in one picture: a model of about 3 billion parameters, squeezed into 2 bits each, on the device, and a larger model in a data center. The rest of this lesson shows why size decides which goes where.

Why size decides where a model runs

A model is mostly its parameters, and each parameter is a number stored in some number of bits. Eight bits make a byte. Four things follow.

The download. Parameters times bytes per parameter. SmolLM2-135M, the language model behind many of this site’s labs, has 134.5 million parameters stored in 16 bits, which is 2 bytes, each: 269 MB. Its weights file on Hugging Face is 269 MB.

The memory. While a model runs, all of it has to sit in the device’s memory, plus working space for the numbers flowing through it. A common rule of thumb from the research group EleutherAI is to add about 20% (Anthony and colleagues, 2023). A long conversation needs more on top, as the KV cache lesson shows.

The speed. To write each new tokenTokenThe unit of text a language model reads and writes. A token can be a whole word, part of a word, a single character, or punctuation, often including a leading space.Taught in TokensOpen in glossary (a word or a piece of a word), a language model uses nearly every one of its parameters once. So for every token, the chip has to read the whole model from its memory. How many bytes a chip can read each second is its memory bandwidthMemory bandwidthHow many bytes a chip can read from its memory each second. A language model reads nearly all of its weights for every token it writes, so bandwidth divided by the model's size caps how fast it can write.Open in glossary. Apple’s M2, the laptop chip this site was tested on, reads 100 GB a second (Apple, 2022). Reading SmolLM2’s 269 MB takes under 3 thousandths of a second, so at best about 370 tokens a second. Llama 2 7B is 50 times bigger, so on the same chip it writes at most about 7 tokens a second.

The battery. All that reading and arithmetic takes energy. A model running on a phone drains the battery and warms the phone, and a bigger model does more work for every token. In the cloud, the data center pays for the electricity.

A data center GPUGPUA graphics processing unit, a chip that does thousands of simple calculations at once. Built for graphics, GPUs also suit the many multiplications inside neural networks, so most AI training and serving uses them or similar chips.Open in glossary, such as the SXM version of NVIDIA’s H100, has 80 GB of memory that it reads at 3,350 GB a second, more than 30 times as fast as the laptop (NVIDIA). A model too big for one GPU is split across several, each holding and reading its own part. That is why the biggest models live in data centers.

Will it fit?

Pick a model, or drag the number of parameters, and choose how many bits each number gets. Besides this site’s own models, the list has three of Meta’s Llama models, whose weights Meta publishes for download, so they can run on any computer that can hold them. The calculator applies the three rules above to a phone, a laptop, and a data center, each described by a few stated numbers, so treat its answers as rough.

# Download, memory, and best-case writing speed on a laptop, from a parameter count.
GB = 1e9                           # file sizes: a billion bytes
LAPTOP_MEMORY = 16 * 2**30         # a "16 GB" laptop holds 16 x 2^30 bytes
LAPTOP_SPEED = 100 * GB            # bytes its memory moves per second (Apple M2)

def on_laptop(params, bits):
    download = params * bits / 8   # bytes: 2 per parameter at 16-bit, half at 4-bit
    memory = 1.2 * download        # plus about 20% working space
    share = memory / LAPTOP_MEMORY
    if share > 1:
        return download, memory, "too big"
    fit = "fits" if share <= 0.5 else "tight"
    speed = LAPTOP_SPEED / download  # each token reads every number once
    return download, memory, f"{fit}, at most {speed:.0f} tokens a second"

models = [("SmolLM2-135M", 134_515_008), ("Llama 2 7B", 6_738_415_616),
          ("Llama 3 70B", 70_553_706_496)]
for name, params in models:
    for bits in (16, 4):
        download, memory, verdict = on_laptop(params, bits)
        print(f"{name:12} {bits:2}-bit: download {download / GB:6.2f} GB, "
              f"memory {memory / GB:6.2f} GB: {verdict}")

SmolLM2-135M 16-bit: download   0.27 GB, memory   0.32 GB: fits, at most 372 tokens a second
SmolLM2-135M  4-bit: download   0.07 GB, memory   0.08 GB: fits, at most 1487 tokens a second
Llama 2 7B   16-bit: download  13.48 GB, memory  16.17 GB: tight, at most 7 tokens a second
Llama 2 7B    4-bit: download   3.37 GB, memory   4.04 GB: fits, at most 30 tokens a second
Llama 3 70B  16-bit: download 141.11 GB, memory 169.33 GB: too big
Llama 3 70B   4-bit: download  35.28 GB, memory  42.33 GB: too big

Every answer came from a multiplication or a division. The download is parameters times bytes per number. The memory is the download plus a fifth. And the best-case writing speed is how fast the memory can be read, divided by the download. A model “fits” when it needs at most half of a device’s memory, since the system and your other apps need the rest, and is “tight” up to all of it.

The speed bands compare with reading. People read about 238 words a minute, about 4 a second (Brysbaert, 2019), and in the tokenizers language models commonly use, every word is at least one token. So under 4 tokens a second is slower than you read, and 20 or more is faster.

The formula is not just theory. Here it is against the 15 models this site runs in your browser; the formula column is parameters times bits each, divided by 8 to get bytes. The real downloads are a little larger: they include small files such as the tokenizer, 8-bit versions keep a few small parts at 32-bit, and 4-bit files also store a scale for each small block of weights. Janus-Pro-1B stores its parts at different precisions, so its formula adds them up part by part.

Why this site runs every model in your browser

Every model on this site, from 24 MB to 1.6 GB, runs inside your browser tab. There are three reasons.

• Privacy you can check. This site is a set of files with no server program of its own. Its pages also tell your browser to refuse connections to anywhere except this site and the Hugging Face Hub, where the model files are hosted. What you type, record, or photograph in a lab is processed in your browser and not sent anywhere.
• No cost per question. A cloud model costs its owner money every time someone uses it. Your browser doing the work costs this site nothing per question, so it needs no accounts and no limits.
• You can look inside. Several labs read the numbers inside a model while it runs, which needs the model’s actual weights on your computer.

What it costs you:

• Downloads. Each model downloads the first time you use it: 24 MB to 1.6 GB, about 3.8 GB for all 15. At 50 megabits a second, SmolLM2’s 272 MB takes about 44 seconds. Every lab asks first and shows the size, and your browser keeps a copy for later visits.
• A slower first answer. After the download, the model still has to be loaded into memory and prepared before it answers.
• Small models only. The largest here, Janus-Pro-1B, has about 1.53 billion parameters. A laptop could hold bigger ones, but every learner would have to download them first.
• Your device does the work. Speed depends on your hardware, as the Deeper section above shows, and on a phone it uses battery.

Smaller numbers: what quantization buys

Storing each number in fewer bits is called quantizationQuantizationStoring a model's numbers in fewer bits, for example as 8-bit or 4-bit integers with a scale, to save memory and speed up inference at some cost in accuracy.Taught in Quantization: smaller numbersOpen in glossary. Going from 16 bits to 4 makes the download and the memory four times smaller. Because writing speed is set by how many bytes are read per token, it also raises the speed ceiling four times. That is how phones fit capable models: at 2 bits per weight, as Apple uses, 3 billion weights take 0.75 GB.

The cost is accuracy. A number stored in 4 bits can take only 16 different values, so every weight gets rounded, and the model’s answers change. By how much depends on the model and the method, so this site measured before choosing:

• SmolLM2-135M: the published 8-bit and 4-bit versions picked a different most likely next token on about a third to nearly half of 24 test prompts, so the site downloads the 16-bit weights.
• CLIP, which matches photos and words: its 8-bit version moved some label probabilities by up to 63 percentage points, so the site uses 16 bits.
• Whisper base.en, which turns speech into text: on 140 test clips its 8-bit version got 6.3% of the words wrong against 5.4% for the 32-bit one, and it still beat the next smaller Whisper model at full precision, so the site uses 8 bits and saves about three quarters of the download.

The quantization lesson shows how the rounding works and why a few unusually large numbers make it worse.