What is a neural network?

Take a photo of a paper check in a banking app, and it reads the handwritten amount, then asks you to confirm it. Think about how hard that would be to write as rules. Everyone writes a seven differently. Some cross it, some slant it, some run it into the next digit. A rule for every handwriting style would never end. Reading messy handwriting from examples is one of the oldest jobs a particular kind of software was built for. It's called a neural network.

Start with the smallest part, a unit, sometimes called a neuron. It takes in a few numbers. It multiplies each one by a weight, which says how much that input counts. Then it adds them up and passes a single number on to the next units. That's all one unit does. On its own, it's about as clever as a calculator doing one sum.²

Units connect to other units, and every connection has a strength: a number that makes a signal count for more or for less as it passes through. Those strengths are what change when the network learns. Nobody sets them by hand. They usually start out as small random numbers, so a brand new network's guesses are nonsense.¹⁴

The units are arranged in layers, each one passing its numbers to the next. The first layer takes in the raw input. For a photo of a check, that's how light or dark each tiny square of the picture is. The last layer gives the guess: this digit is probably a seven. In between are hidden layers, named that because you never see what they pass along. A small network might have a handful of layers, and many modern ones have dozens.¹²

Learning is a loop. The network looks at an example that a person has already labeled, a handwritten digit with the right answer attached, and makes a guess. It measures how far off the guess was. Then it nudges every connection strength a tiny amount in the direction that would have made that guess less wrong. One example changes almost nothing. Repeat it over thousands or millions of examples, and the strengths settle into values that work. In 1989, researchers at Bell Labs trained a network this way on about 7,300 handwritten digits from zip codes on real mail. It learned to read new ones it had never seen.³

The name does come from the brain. Early researchers borrowed the picture of cells that pass signals along connections of different strengths. But the resemblance stops early. A unit is a line of arithmetic. A brain cell is a living thing, and scientists are still working out the rules brains use to learn. So when you hear that a neural network thinks like a brain, it's fair to translate that as: it was loosely inspired by one.¹⁵

Neural networks sit under nearly every modern AI system you use, from voice assistants to photo search to chatbots. The deep in deep learning just means a network with many layers. The chatbots are enormous versions, with billions of connection strengths. The next time your phone opens because it saw your face, one of these is doing the looking. Topic 100 shows how.¹