What is a GPU, and why does AI need so many?

A video game on your screen is made of millions of tiny dots, and the game redraws them many times every second. Each dot needs its own small calculation: what color, how bright. The chip built for that job is called a graphics processing unit (GPU), the heart of a graphics card. Years later, it turned out to suit artificial intelligence (AI) remarkably well.¹

Every computer has a main processor, the central processing unit (CPU). It's built to handle a few things at a time, one after another, very fast, and to switch between all kinds of jobs. A GPU is built differently. It has hundreds or thousands of smaller cores, and each one does a simple calculation, all at the same time. That's called parallel processing: one big task broken into many small pieces that are worked on at once.¹

Picture a mountain of onions to chop. One brilliant chef is fast, but there's only one knife. A hundred ordinary cooks, each with a small pile, finish long before the chef does. The chef is the main processor, great for a recipe with many different steps. The hundred cooks are the GPU. They're far faster when the same simple job has to be done over and over. Same onions, very different afternoon.

AI models are made of billions of adjustable numbers called parameters. Training a model means nudging all of those numbers, again and again, using the same simple multiplications and additions. Most of those sums don't depend on each other, so they can all be done at once. Billions of small sums, over and over. That's the mountain of onions a GPU was built for.²

Even a GPU has limits. The largest models are trained across thousands of GPUs linked together, each one handling part of the work and sharing results with the rest. And training is only half of it. Every time you ask a chatbot something, chips like these work out the answer, a step called inference. With millions of people asking at once, that takes a lot of hardware.²³⁶

That hardware explains two things you may have read about. Big models are expensive, and only the best-funded organizations can afford to build the largest ones. And data centers use a lot of electricity. The International Energy Agency estimates that in 2024, servers built for AI, packed with chips like these, used about 15 percent of all the electricity that data centers used worldwide. It expects that share to keep growing.⁴⁵

So the next time you see a headline about a company buying tens of thousands of chips, you'll know what they're for: doing a huge number of small sums at the same time, so a model can learn, and then answer. Your own phone and laptop have a small graphics processor too, usually built into the main chip. It's the same idea, in miniature. Do you know what yours is called?