THE IDEA

A heartbeat of data

A systolic array is a grid of simple processing elements (PEs). On every clock tick, each PE multiplies the two numbers arriving from its neighbors, adds the product to its total, and passes the numbers on. Data flows through the grid in waves, like blood pumped by a heart. The idea was described by H. T. Kung and Charles Leiserson in the late 1970s.

Each number is read from memory once, then reused by a whole row or column of PEs.

WATCH IT

Multiply two matrices, tick by tick

Here one matrix enters from the left and the other from the top, each row and column delayed by one tick. Each PE ends up holding one entry of the answer.

An N × N array does N² MACs every tick, after a short start-up wave.

THE TPU

Google's Tensor Processing Unit

🟨

The simulator showed the classic textbook design. The TPU's version differs a little: weights are loaded and held in the cells, inputs flow in from the left, and partial sums flow down to accumulators below the array.

2013 Google projects that 3 minutes a day of neural-network voice search per user would force it to double its data centers

2015 TPU v1 goes into Google data centers: a 256 × 256 array of 8-bit MACs (65,536) for inference

2016 Google announces the TPU publicly; a detailed paper follows in 2017

2017 onward Later TPUs add training, BF16, HBM and links that join thousands of chips into 'pods'

The TPU showed that a custom AI chip could beat contemporary CPUs and GPUs by large factors in performance per watt.

WHY IT WINS

Short wires, few fetches

📏

♻️

Reuse

A weight loaded into a PE is used for many inputs in a row

📏

Short wires

Each PE only talks to its neighbors, so wires stay short and cheap

🎯

Simple control

No caches or branch logic per PE: almost all area is math

A systolic array turns the memory wall into a short hop between neighbors.

THE WEAK SPOT

Great for big matrices, not everything

🧩

A systolic array shines when matrices are large and regular. Small or oddly shaped work leaves many PEs idle, and anything that is not a matrix multiply runs on separate vector or scalar units. Designers choose the array size to fit the models they expect.

The array's size is a bet on the shape of future AI models.

UNIT 6 STUDY COMPLETE

💓

Ready for the Fab Challenge?

You've watched data pulse through a systolic array, seen why it reuses every number it fetches, and met the TPU built around one.