THE PROBLEM

Feeding thousands of math units

An AI chip that does a quadrillion operations per second needs terabytes of data per second. Ordinary graphics memory on a circuit board, connected by long traces, can't keep up within a sensible power budget. The answer is High Bandwidth Memory (HBM).

HBM exists because of the memory wall.

BUILD IT

A stack of DRAM dies

An HBM stack is a base logic die with 8, 12 or 16 DRAM dies on top. Each die is thinned to a few tens of micrometres and connected to the next by thousands of through-silicon vias (TSVs): copper-filled holes that run straight through the silicon.

TSVs turn a stack of chips into one tall, very wide memory.

WIDE, NOT FAST

Bandwidth from width

Each HBM stack has a 1,024-bit interface (2,048 bits in HBM4), far wider than the 32 bits of a GDDR chip. Many slower wires use less energy per bit than a few very fast ones. Bandwidth = width × data rate per pin × number of stacks.

Eight HBM3e stacks give a B200 about 8 TB/s.

THE GENERATIONS

HBM keeps climbing

📈

HBM (2013–15) First standard; AMD's Fiji GPU was the first to use it

HBM2 / HBM2E Used in NVIDIA's P100, V100 and A100 and Google's TPUs

HBM3 / HBM3E About 0.8 to over 1.2 TB/s per stack: H100, H200, B200, MI300

HBM4 2,048-bit interface, standardized in 2025, with stacks up to 16 dies high

Each generation roughly doubles bandwidth or capacity per stack.

WHO MAKES IT

Three memory makers

🏭

Three companies supply almost all HBM: SK hynix (the leader in recent generations), Samsung and Micron, with China's CXMT trying to catch up. HBM is hard to make: thinning, TSVs and stacking all lower yield, and Micron says HBM3E takes about three times the wafers of DDR5 for the same number of bits.

HBM supply is one of the main limits on how many AI chips the world can build.

UNIT 11 STUDY COMPLETE

🥞

Ready for the Fab Challenge?

You've built an HBM stack from thinned DRAM dies and TSVs, and worked out how its very wide interface delivers terabytes per second.