ONE CHIP

Hundreds of watts on a few cm²

🔥

Desktop CPU Roughly 65–250 W

NVIDIA A100 (2020) Up to 400 W

NVIDIA H100 SXM (2022) Up to 700 W

NVIDIA B200 (2024) Around 1,000 W

NVIDIA Rubin (2026) Reported at around 1,800 W

AI chip power has roughly doubled every couple of generations, because performance per chip grew faster than efficiency.

WHY SO HOT?

Power = switching × voltage²

Dynamic power is roughly P = α·C·V²·f: how often transistors switch (α), their capacitance (C), the voltage (V) squared and the clock (f). Billions of transistors switching billions of times a second add up, and the voltage has stopped falling much with each node.

Dennard scaling ended around 2006, so more transistors now means more heat.

ADD IT UP

From chip to campus

Multiply by the number of GPUs, add CPUs, networking and memory, and then add cooling and power-conversion overhead. The overhead is measured by PUE (power usage effectiveness): total facility power divided by IT power.

A 100,000-GPU cluster can draw well over 100 MW, as much as a small city.

ONE RACK

A rack that needs its own substation

NVIDIA's GB200 NVL72 rack holds 72 Blackwell GPUs and 36 Grace CPUs and draws roughly 120 kW. A typical older data-center rack drew 5–15 kW. Racks this dense need liquid cooling and much heavier power feeds.

AI racks are about ten times denser in power than traditional server racks.

THE BIG PICTURE

Data centers and the grid

🌍

The International Energy Agency estimated that data centers used about 415 TWh in 2024, around 1.5% of the world's electricity. Its 2026 update reportedly puts 2025 at about 485 TWh and projects about 950 TWh by 2030, around 3% of the world's electricity, with AI as the main driver.

Energy may become the main limit on how fast AI can grow.

UNIT 14 STUDY COMPLETE

🔌

Ready for the Fab Challenge?

You've added up AI power from one chip to a whole data center, and seen how much electricity the world's data centers use.