A RULE OF THUMB

Training ≈ 6 × N × D

📐

Training a transformer takes roughly 6 × N × D floating-point operations, where N is the number of parameters and D the number of training tokens. GPT-3 (175 billion parameters, about 300 billion tokens) needed roughly 3 × 10²³ operations.

Double the model or double the data and the training compute doubles.

ESTIMATE IT

From FLOPs to megawatt-hours

Divide the total operations by what each GPU actually delivers (often 30–50% of its peak), multiply by its power and by the PUE, and you get the energy of a training run.

Patterson and colleagues estimated GPT-3's training at about 1,300 MWh: roughly a year's electricity for over a hundred US homes.

ANSWERING

The energy of one question

Inference takes about 2 × N operations per token. One answer is tiny compared with training, but billions of questions a day add up, so over a model's life inference can use more energy than training. Google measured about 0.24 Wh for a median Gemini text prompt in 2025, and Epoch AI estimated about 0.3 Wh for a typical ChatGPT query, a tenth of an older 3 Wh figure. Long answers and reasoning models can use much more.

Be careful with per-query numbers: they depend heavily on what is assumed.

IMPROVING

Getting more for each joule

📉

🔽

Lower precision

FP8 and FP4 cut energy per operation

✂️

Smaller models

Distillation and mixture-of-experts use only part of the weights per token

📦

Batching

Serving many users at once reuses each weight read

Energy per answer has fallen quickly, even as total demand has risen.

CARBON

It depends on the grid

The same megawatt-hour can produce very different carbon emissions depending on whether it comes from coal, gas, hydro, nuclear, wind or solar. That is why AI companies sign deals for clean power and why data centers are built near cheap, clean electricity. Making the chips also has a carbon cost.

Energy is a chip problem; carbon is also a grid problem.

UNIT 16 STUDY COMPLETE

🧮

Ready for the Fab Challenge?

You've estimated the energy of a training run from the model size and data, and seen why the energy of a single answer is so hard to pin down.