GO BIG
A chip the size of a wafer
Instead of cutting a wafer into hundreds of dies, Cerebras keeps it whole. Its WSE-3 (2024) covers about 46,000 mm², roughly 57 times an H100, with about 4 trillion transistors, 900,000 cores and 44 GB of on-chip SRAM.
HOW IS THAT POSSIBLE?
Living with defects
Every wafer has defects, so a wafer-sized chip is never perfect. Cerebras builds in spare cores and routes around the bad ones. Wiring between the reticle-sized fields of the wafer is added across the scribe lines that would normally be cut.
DATAFLOW
Let the data set the pace
In a dataflow chip, a model's layers are mapped onto different regions of the chip, and data streams from one region to the next as soon as it is ready. Companies such as SambaNova build reconfigurable dataflow chips; Groq's LPU uses a fixed, compiler-planned schedule with all weights in on-chip SRAM.
WHO'S WHO
Different bets
THE TRADE-OFFS
SRAM is fast but small
On-chip SRAM is far faster than HBM but holds far less per mm². A model with hundreds of billions of weights may need many SRAM-only chips working together. Each design trades capacity, speed, cost and how easy it is to program.
UNIT 7 STUDY COMPLETE
Ready for the Fab Challenge?
You've met chips that use a whole wafer, keep everything in on-chip SRAM, or lay a model out across the silicon.