This one difference explains almost everything: why AI runs on GPUs, why GPUs use more power, and why every computer still needs a CPU. Each section below runs the same task on both, side by side. Press play to see it.
Think of a picture made of 2,400 small squares. A CPU paints it with 16 skilled painters. Each one is fast, but there are only 16. A GPU uses thousands of painters. Each one is slower and can only do simple work, but all of them paint at once.
Use a CPU core for one hard problem. Use thousands of GPU cores for thousands of small, identical calculations.
A chip has a fixed amount of space and can use only so much power. Every part spent on one job cannot be used for another. CPU and GPU designers spend that space in very different ways, because they expect very different work.
GPU cores work in groups of 32 that must all run the same step at the same time. If the code makes a decision and some of the 32 need a different path, the group runs one path while the others wait, then runs the other path. Code full of decisions leaves most of a GPU waiting. A CPU core follows each decision on its own.
A CPU is built to finish one chain of steps as fast as possible. A GPU is built to finish as much total work as possible. Each one is fast at its own kind of work and slow at the other.
Only partly. A laptop has a few cores but many more apps. Each core gives an app a few milliseconds, then moves to the next one. This switching is so fast that all apps look like they run together. Click "Slowed down" to see each switch.
A CPU runs many different jobs by switching between them. A GPU runs one simple job on thousands of numbers at the same time. Apps need the first. Graphics and AI need the second.
An AI model is made of large tables of numbers. To write each word of an answer, it multiplies two numbers and adds the result to a total, again and again, billions of times. Each square below is one such total. No square depends on another, so all of them can be calculated together.
AI does not need a few powerful cores. It needs one simple calculation repeated billions of times at once. GPUs were built for this kind of work in games, and AI needs the same thing.
Small models run on a normal laptop CPU without any problem. The speed gap grows as the model gets bigger. Choose a model size and press play. Both sides write the same answer.
For one person testing a small model, a CPU is enough. For large models, or many users at once, it is too slow. The next section explains the main reason.
Cores can only work as fast as numbers reach them. For every word it writes, an AI model reads all of its numbers from memory once. A medium model holds about 16 GB, so it reads 16 GB for each word. Memory speed matters as much as core count.
A GPU has about 80 billion transistors and thousands of cores. When it works, most of them switch on and off billions of times per second, and its memory runs at very high speed too. A CPU uses much of its chip for control and cache, so fewer parts are busy at once. This is why a GPU draws more power. It finishes AI work much faster, though, so it uses less energy for each task.
Compare a bus and a car. A bus uses more fuel per hour, but less fuel per passenger. An 8-GPU server can draw about 10 kW, roughly what six home air conditioners use. This is why AI data centres need so much power and cooling.
A GPU does not replace a CPU. Every computer, including an AI server, has a CPU in control. A GPU takes the heavy, repetitive maths. Choose the right chip for each task below.
A CPU decides what to do. A GPU does the heavy maths.
Hardware figures: NVIDIA H100 SXM (16,896 CUDA cores, about 80 billion transistors, up to 700 W, 3.35 TB/s HBM3), a 16-core desktop CPU with dual-channel DDR5 (about 90 GB/s), servers up to 192 cores. Speeds in the animations and the work-per-watt range are illustrative, to show the idea, not benchmarks.