Compute and energy

How much electricity does a chatbot use to answer one question? An often-repeated figure from 2023 is about 3 watt-hours, ten times a web search. Google measured 0.24 watt-hours for its median Gemini text prompt in 2025. OpenAI’s chief executive has said 0.34. An estimate for a question about a very long document is almost 40.

These numbers are not all wrong, and they are not all comparable. Each counts different things, for different models, in different years. This lesson builds the arithmetic underneath them: what training a model takes, what answering a prompt takes, and what all of it adds up to. Then you can tell what a number means before you decide what to make of it.

Training: counting operations

In How LLMs are trained you met the rule that training a model with $N$ parameters on $D$ tokens takes about $6ND$ operations: about 2 per parameter per token on the way forward and 4 on the way back. Each is a floating point operationFLOPA floating point operation, one multiplication or addition of numbers with fractional parts. Training and running models are measured in how many of these they take, and chips in how many they do per second (FLOP/s).Open in glossary (FLOP), one multiplication or addition of decimal numbers. The counts get large quickly.

DeepSeek-V3 is a mixture of expertsMixture of expertsAn architecture with many alternative sub-networks (experts) and a router that runs only a few of them for each token, so a model can store many parameters while using few per token.Taught in Mixture of expertsOpen in glossary: each token passes through 37 billion of its 671 billion parameters, so only those count, and it trained with about a twelfth of Llama 3.1 405B’s operations. GPT-4’s figure is uncertain because OpenAI has not published the inputs. Across the field, Epoch AI estimates that the compute used to train notable models grew 4 to 5 times a year from 2010 to May 2024 (Sevilla and Roldán, 2024). Its February 2026 update puts the growth for frontier language models at about 5 times a year since 2020, and the cost of training them at about 3.5 times a year (Epoch AI).

From operations to GPU hours

A data center GPUGPUA graphics processing unit, a chip that does thousands of simple calculations at once. Built for graphics, GPUs also suit the many multiplications inside neural networks, so most AI training and serving uses them or similar chips.Taught in Where your AI runsOpen in glossary such as NVIDIA’s H100 can do about 989 trillion 16-bit operations a second. NVIDIA lists 1,979 trillion, but that rate uses a sparsity feature that skips half of the numbers, which ordinary training does not use (NVIDIA).

No real run keeps a GPU that busy. Chips wait for numbers to arrive from memory and from each other, and long runs stop and restart after failures. The share of the peak a run actually spends on the model’s own operations is its model FLOPs utilizationModel FLOPs utilizationThe share of a chip's peak arithmetic rate that a run actually spends on the model's own operations. Meta reported 38 to 43 percent for the main training setups of Llama 3.1 405B.Open in glossary. Meta reported 38% to 43% for Llama 3.1 405B’s main training setups. So

and the run lasts GPU hours divided by the number of GPUs. GPT-3 ran on about 10,000 V100 GPUs for about 14.8 days, by the estimate of Patterson and colleagues (2021): OpenAI measured 24.6 trillion operations a second per GPU, a fifth of the V100’s peak of 125 trillion, and the GPU count came from NVIDIA. BLOOM’s 1.08 million A100 hours over 118 days work out to about a third of the A100’s peak (Luccioni and colleagues, 2022).

From GPU hours to electricity

A GPU draws up to 300 W (V100), 400 W (A100), or 700 W (H100) while it works. The rest of each server, its processors, memory, fans, and network, draws more. The building adds cooling and power conversion, measured by its power usage effectivenessPower usage effectivenessA data center's total electricity divided by what its computers use. A value of 1.0 would mean no overhead for cooling and power conversion; Google reported a fleet average of 1.09 in 2025.Open in glossary (PUE): the whole site’s electricity divided by what its computers use. OpenAI reported 1.10 for the data center that trained GPT-3; Google reported a fleet average of 1.09 in 2025.

The published training figures:

• GPT-3: 1,287 megawatt-hours (MWh), estimated by Patterson and colleagues from OpenAI’s measured 330 W per GPU (including its share of the server), the 1.10 overhead, and the 14.8 days.
• BLOOM: 433 MWh for the GPUs at their 400 W rating, without the building. From the cluster’s measured power when idle, the authors estimated another 257 MWh for the time the machines were on.
• Llama 3.1 405B: 30.84 million H100 hours at 700 W each, which Meta describes as peak power adjusted for the data center’s overhead (model card): about 21,600 MWh. Meta counts that as 8,930 tonnes of CO₂-equivalent by the average mix of the local grids, and as zero against the renewable electricity it buys to match its use. Both are standard ways to account for emissions; they answer different questions.
• Grok 4 (2025), estimated from outside: xAI has not published its training figures. Epoch AI estimated about 246 million H100 hours and 310 GWh (310,000 MWh), counting each GPU at three quarters of its 700 W peak, the rest of the hardware, such as servers and network switches, at about as much again, and 1.2 for the building. It warns that the hours rest on vague public statements from xAI, so the uncertainty is large (Epoch AI, September 2025). That is about 14 times Llama 3.1 405B’s figure, but counted more widely: at Meta’s 700 W per GPU, the same hours come to about 172 GWh, 8 times Llama’s.

For scale, an average U.S. home used 10,791 kWh of electricity in 2022 (EIA). GPT-3’s training used about as much as 120 homes in a year, Llama 3.1 405B’s about 2,000, and Grok 4’s, by Epoch AI’s estimate, about 29,000.

Electricity is a small part of what training costs in money. Epoch AI estimated, with a confidence it rates as speculative, GPT-4’s final training run at about $40 million, counting the share of the hardware’s value used up during the run plus its electricity, and the hardware itself at about $800 million to buy. For four frontier models it studied in detail, energy was 2% to 6% of the whole development cost, staff included; the chips and the staff were the largest parts (Cottier and colleagues, 2024). The authors rate single estimates like this as uncertain from about a third to four times the figure. DeepSeek reported 2.788 million H800 GPU hours for DeepSeek-V3, $5.6 million at an assumed rental price of $2 an hour, and noted that this leaves out the research and experiments before the final run. Training the reasoning models R1-Zero and R1 from it took about 147,000 more H800 hours, including making their fine-tuning data, $294,000 at the same price, according to the supplementary information of its peer-reviewed paper (DeepSeek-AI, 2025). For Grok 4, Epoch AI’s median estimate, from rental prices, is $490 million.

Try it: a training and serving calculator

Pick a published run, or set your own. The calculator follows the steps above with stated assumptions: each GPU at its rated maximum power, times 1.1 for the building. Under each preset it shows what was actually reported, so you can see how close a simple model gets.

# Training: operations -> GPU hours -> days -> energy.
PUE = 1.1                                 # building overhead: cooling, power conversion

def train(params, tokens, peak, utilization, gpus, watts):
    ops = 6 * params * tokens             # 2 forward + 4 backward per parameter per token
    gpu_hours = ops / (peak * utilization) / 3600
    days = gpu_hours / gpus / 24
    mwh = gpu_hours * watts * PUE / 1e6   # watt-hours to megawatt-hours
    return ops, gpu_hours, days, mwh

# GPT-3 on 10,000 V100s at OpenAI's measured 24.6 of 125 trillion operations a second.
ops, hours, days, mwh = train(174.6e9, 300e9, 125e12, 24.6 / 125, 10_000, 300)
print(f"GPT-3: {ops:.3g} operations, {days:.1f} days, {mwh:,.0f} MWh at 300 W x {PUE}")
print(f"  at the measured 330 W x 1.10: {hours * 330 * 1.10 / 1e6:,.0f} MWh")

# Llama 3.1 405B: what share of the H100's peak do Meta's GPU hours imply?
ops = 6 * 405.85e9 * 15.6e12
print(f"Llama 3.1 405B: {ops:.2g} operations, {ops / (30.84e6 * 3600 * 989.5e12):.1%} of peak")

# Serving: Epoch AI's estimate for one GPT-4o query, from its stated assumptions.
query_ops = 2 * 100e9 * 500               # 2 per active parameter, 500 output tokens
seconds = query_ops / (989.5e12 * 0.10)   # an H100 at 10% utilization
print(f"one query: {seconds * 1500 * 0.7 / 3600:.2f} Wh")  # 1,500 W all in, 70% drawn

GPT-3: 3.14e+23 operations, 14.8 days, 1,171 MWh at 300 W x 1.1
  at the measured 330 W x 1.10: 1,288 MWh
Llama 3.1 405B: 3.8e+25 operations, 34.6% of peak
one query: 0.29 Wh

At the presets, the calculator’s energy lands within 10% of each published figure, and the figures themselves differ in what they count: GPT-3’s includes the servers and the building, BLOOM’s 433 MWh only the GPUs, Meta’s the GPUs at peak power with the building. Three lessons from the calculator:

• More GPUs finish sooner but do not save energy. They share the same operations. Energy falls only with fewer operations, a better-used chip, or a chip that does more per joule.
• Utilization is far below peak. Meta’s reported GPU hours imply about 35% of the H100’s peak over the whole run, below the 38% to 43% it measured on its main setups. The model card does not say why; candidates include work that $6ND$ leaves out, such as attention over the 128,000-token sequences trained near the end, and restarts: Meta counted 466 interruptions in 54 days.
• Newer chips do more per joule. At full power a V100 spends about 2.4 trillionths of a joule per operation and an H100 about 0.71, more than three times less.

Serving: energy per answer

Training happens once. InferenceInferenceRunning a trained model to get an output, as opposed to training it. Every prompt a chatbot answers is inference, and over a popular model's life it can use more energy than the training did.Open in glossary, running the trained model, happens every time someone sends a prompt. Reading or writing each token takes about 2 operations per active parameter, so a 500-token answer from a model with 100 billion active parameters takes about $10^{14}$ operations.

Serving uses chips less efficiently than training. While writing, each new token waits for the model’s weights to stream in from memory, the speed limit you met in Where your AI runs. Providers answer many people’s requests together to reuse each read, and keep spare machines ready for busy hours. Here are the per-prompt figures behind this lesson’s first paragraph:

Sources: Elsworth and colleagues (Google, 2025), Epoch AI (You, 2025), Oviedo and colleagues (2026), Altman (2025), de Vries (2023), Luccioni and colleagues (2022).

Three things explain most of the spread:

• The boundary. Google’s two numbers for its median prompt differ by 2.4 times because of what they count, and because the smaller one samples only its most efficient data centers.
• The work. A median prompt is short. A long document to read, or a long chain of reasoning to write, multiplies the operations: in Oviedo and colleagues’ estimate, an answer 15 times longer takes 13 times the energy; see Thinking longer and Long context in practice. The International Energy Agency adds that video generation, reasoning, and agentic tasks can take hundreds or thousands of times the energy of a simple text query (IEA, April 2026).
• The year. Google reported that its median prompt’s energy fell 33 times from May 2024 to May 2025, mostly from improvements to its models and software.

Over a model’s life, serving can outweigh training. Across all of its machine learning, Google measured that about three fifths of the energy went to inference in 2019 to 2021, and two fifths to training (Patterson and colleagues, 2022).

Why the price keeps falling

Providers charge per million tokens. Epoch AI tracked the cheapest price for reaching a fixed benchmark score and found it falling between 9 and 900 times a year, depending on the benchmark and the score; for matching GPT-4 on a set of PhD-level science questions, about 40 times a year. The fastest drops were in the most recent year, so they may not last. Where the fall comes from:

• Smaller models trained for longer, and taught by larger ones through knowledge distillationKnowledge distillationTraining a smaller student model to reproduce a larger teacher model's outputs, so the student learns from the teacher's predictions instead of, or as well as, human labels.Taught in Depth from a single imageOpen in glossary, reach scores that once took much bigger models (see “Beyond compute-optimal” in How LLMs are trained).
• Fewer bits per number (Quantization) and mixtures of experts cut the bytes and operations per token.
• Reuse: the KV cache avoids recomputing a conversation, and serving many requests together reuses each read of the weights.
• Hardware that does more operations per joule and per dollar.

A price is not a cost. It also reflects competition and margins, which providers do not publish.

Data centers and the grid

All of this runs in data centers, which also serve search, video, storage, and the rest of the internet. The IEA estimated that data centers used about 415 terawatt-hours (TWh) in 2024, about 1.5% of the world’s electricity, with the United States at 45%, China 25%, and Europe 15% (IEA, April 2025). Its April 2026 update put 2025 at 485 TWh, up 17% in a year, while the electricity used by AI-focused data centers grew 50%. It projects about 950 TWh by 2030, around 3% of the world’s electricity. The range is wide: in the 2025 report, the scenarios for 2035 spanned 700 to 1,700 TWh.

In the 2025 report, that growth was about a tenth of the world’s growth in electricity demand to 2030, less than from air conditioning or electric vehicles, but it is concentrated: a typical AI-focused data center uses as much electricity as 100,000 households, and the largest under construction 20 times as much. In the United States, data centers used 4.4% of the electricity in 2023 (DOE and LBNL, 2024), and Lawrence Berkeley National Laboratory projects 9.5% to 15.3% by 2030 (LBNL, June 2026).

Both trends are true at once. The energy per task is falling fast; the IEA says by at least ten times a year recently. Total use is rising anyway, because use grows faster and new tasks need more. Cheaper answers can lead to more answers.

Water

Data centers use water mostly for cooling, and power plants use water to make the electricity. Published figures per answer differ by more than a hundred times, mostly because they count different things:

• Google: 0.26 mL for the median Gemini text prompt, the water its data centers consume for cooling (May 2025; 0.12 mL by the narrower method).
• Mistral: 45 mL for a 400-token answer from its Le Chat assistant, from a life-cycle study with the consultancy Carbone 4 and the French agency ADEME, reviewed by two other consultancies, that also counts upstream impacts such as making the servers (Mistral, July 2025).
• Li and colleagues: one 500 mL bottle per 10 to 50 medium-length GPT-3 answers, so 10 to 50 mL each, counting the cooling water in Microsoft’s U.S. data centers and the water used to generate their electricity. They also estimated 5.4 million liters for training GPT-3, of which 700,000 evaporated at the data center (Li and colleagues, 2025). Both are estimates from outside.

Where matters as well as how much: Li and colleagues show that the water per unit of electricity varies by place and by time of day, and Google says it limits water use in watersheds under stress.