THREE APPROACHES

Flexible vs. fast

🧑‍💼

CPU

A few powerful cores that run any program, with big caches and clever tricks for branchy code

👥

GPU

Thousands of simple lanes that all run the same instruction on different data

🎯

ASIC

An application-specific chip with hardware built for one job, such as matrix multiplication

The more specialized the chip, the more of its silicon does useful math, and the less it can do anything else.

PARALLEL WORK

Many hands make light work

Every MAC in a matrix multiply is independent, so thousands can run at once. But any part of a program that must run step by step limits the speed-up. That limit is called Amdahl's law.

AI is almost perfectly parallel, which is why GPUs beat CPUs by so much at it.

WHERE THE SILICON GOES

Control vs. compute

A CPU spends much of its area on caches, branch prediction and out-of-order logic that make one thread fast. A GPU spends most of its area on arithmetic units. An AI ASIC goes further and drops graphics hardware and general features entirely.

Same transistors, different budgets: an AI chip spends them on math and memory bandwidth.

THE ACCIDENTAL AI CHIP

Why GPUs got there first

🎮

GPUs were built to shade millions of pixels in parallel for video games. In 2007 NVIDIA released CUDA, which let programmers use that parallel hardware for any math. When deep learning took off around 2012, GPUs and their software were ready.

Software matters as much as silicon: CUDA's head start is a big reason NVIDIA leads today.

THE TRADE-OFF

Specialize or stay flexible?

⚖️

CPU Runs anything; slowest per watt at AI math

GPU Programmable and fast; huge software ecosystem

ASIC Best performance per watt for its target; costly to design and can fall behind if AI models change

FPGA Rewirable logic; good for prototypes and niche low-latency jobs

Companies with huge, predictable workloads, like Google and Amazon, design their own ASICs.

UNIT 2 STUDY COMPLETE

⚙️

Ready for the Fab Challenge?

You've compared a few strong CPU cores with thousands of simple GPU lanes and custom ASICs, and seen why parallel work favors the many.