An AI model is mostly one operation repeated: multiply a big table of weights by the numbers coming in. Follow one such multiplication, for one layer of an 8B model, through an NVIDIA H100. Press Next to move a step, or Play to watch it run.
H100 SXM figures from NVIDIA: 132 SMs, 128 CUDA cores and 4 Tensor Cores per SM, 256 KB of registers and up to 228 KB of shared memory per SM, 50 MB of L2 cache, 80 GB of HBM3 at 3.35 TB/s. The steps are simplified; real software fuses several of them.
The same job on four kinds of processor: multiply two 8,192 × 8,192 tables of numbers, about 1.1 trillion multiply-and-add steps. This is the kind of work every layer of an AI model does, over and over. Times are at each processor's published peak speed.
Peaks: 64-core server CPU about 5 TFLOPS in FP32 (illustrative, varies by model); H100 SXM CUDA cores 67 TFLOPS FP32; H100 Tensor Cores 989 TFLOPS in BF16 and 1,979 TFLOPS in FP8 (dense, NVIDIA). Real code reaches a share of peak; the order does not change.