How does my phone recognize my face?

You pick up your phone in a dark room, wearing glasses you didn't have on when you set it up, and it opens anyway. If it were comparing you to a photo, that would be hard to explain. The light is different. Your face is at an angle, and the glasses are new. What it compares is something else: a set of numbers that describe your face.

When you first turn on face recognition, the phone usually asks you to move your head slowly, so it can see your face from several angles. From those views it builds what Google calls a face model and Apple calls a mathematical representation. Both mean a long list of numbers that captures what makes your face yours. You couldn't read them, and they don't look like a face.¹³

Then the pictures go. Apple says the images of your face are thrown away once the numbers are worked out. Google says the same about its Pixel phones: the images used to make the face model aren't stored. What stays is the list of numbers, and only that.¹³

Every time you look at the phone, it takes a fresh look and runs it through a neural network, the kind of layered, trained system from topic 52. The network turns this new view into a new list of numbers. Then the phone compares the two lists with one short question: are these close enough to be the same face? Two views of you, in different light, should land close together. A stranger's face should land far away. Apple says its networks were trained on over a billion images, gathered with people's informed consent, so they learn which differences matter and which ones, like lighting, don't.¹

Phones don't all look in the same way. Apple's iPhones project thousands of invisible dots onto your face and read them with an infrared camera, which builds a map of its depth. That's one reason they work in the dark. Other phones, including Google's Pixel 8 from 2023, work from the ordinary front camera. Both end in the same place, a list of numbers to compare, but they gather the picture differently. So the details depend on the phone in your hand.¹²⁴

The numbers aren't sent anywhere to be matched. Apple keeps them in a separate, protected part of the phone's chip, which it calls the Secure Enclave, and says they never leave the device and aren't included in backups. Google says the face model is stored securely on the phone and never leaves it. The match happens right there, in your hand.¹³

Faces change. You grow a beard or swap your glasses. Apple says its system updates the stored numbers over time to keep pace. If it nearly recognizes you, and you type your passcode straight after, it can add your new look to what it keeps. That's part of why a new beard doesn't lock you out for long.¹

Whether face recognition suits you is your call. Google's own help page says it may be less secure than a strong personal identification number (PIN), pattern, or password. Apple suggests a passcode for anyone concerned about a look-alike sibling or a young child. If you share a home with someone who looks a lot like you, you might pick the passcode instead, and that's a perfectly good choice.²³