Your eyes vs. computer vision

Almost half of your brain is devoted to the visual system — and a surprising amount of brainpower is used to identify faces. Facial recognition programs try to recreate that lightning-fast human cognition. It’s a work in progress.

I’ve just seen a face

In a fifth of a second, your brain can decide whether a photo shows a face or some other object. Some neurons in the visual cortex — aptly called “face cells” — are dedicated solely to faces. They register different features and the spatial relationship between them.

Making sense of faces

Did you get it?

We’re exceptionally good at recognizing faces. It takes only a fraction of a second to recognize a loved one or a celebrity.

Can you recognize this face in a flash?

Let’s make it harder

Who did you see?

If you guessed Marilyn Monroe, you got it. But it’s much harder to recognize a face if you take away a feature. That’s because, to your brain, a face is not a single object, but a combination of features. Most important are the eyes, mouth, and — surprisingly — eyebrows.

Can you recognize this face without one feature?

It’s easier with eyebrows

How about now?

Here’s the same person’s face, but with no features removed.

Now can you guess who this is?

Let’s make it even harder

If you guessed Kim Kardashian, you’re right. Though she was missing a feature, your brain still knew it was looking at a face.

But when a face is upside down, your face-recognizing cells have trouble. Key landmarks are out of place, so your brain has to draw on other parts of its visual cortex. Upside-down faces confuse AI, too. One computer thought an upside-down image of Kardashian was another celebrity entirely.

AI guessed Ozzy Osborne

Talk about a crazy train! An upside-down photo of Kim Kardashian confused AI so much, it thought it was looking at Ozzy Osbourne.

According to AI, who is this?

Like the brain, facial recognition software identifies features and measures the distance between them. Often, it’s better than humans. Humans can instantly recognize thousands of faces. AI can recognize millions.

But while the human brain can work around a missing feature, AI has more trouble. A computer saw these images and made its best guess about whom it was seeing.

The risk of false arrest

If AI has this much trouble spotting celebrities, how does it do in real-life situations? So far, not great. Some police departments have piloted facial recognition software and found disturbing results: one program tested by the London metropolitan police made false identifications 80 percent of the time. For now, it still takes humans to guard civil rights.

Know what I mean?

When you listen to someone speak, your brain makes educated guesses about the words that are coming next. Artificial intelligence uses a similar process to predict the words you’ll text next on your phone.

One of these words could save your life

Early human brains adapted to connect speech with sensory events. Specific neurons in the auditory cortex evolved to respond to different phonemes — small units of sound such as “f,” “t,” or “th” — because anticipating the next word could save precious seconds in life-or-death situations.

So what happens when we type?

It’s easy with context

In written language, context is still important. AI evaluates the word you’ve typed with your thumbs and suggests different words that could work better. (In this case, it might depend on how you feel about choirs.)

What word comes next?

How AI makes connections

To predict text, AI also tries to understand the relationships between words. But AI isn’t hardwired like the brain; it has to learn in a different way. The iPhone and Gmail messaging apps try to guess the next word you’ll type by training on millions of documents and learning the correlations among letters, words, and sentences.

Predicting what comes next

Here’s a game you can play with your friends: Start a text message with the words “Do you want to meet at 6 for a ” — don’t forget the space at the end — and see what the AI suggests for the next word in the sequence. Depending on your message history, your phone will choose three words that might be different from anyone else’s.

The Christopher Walken experiment

Sometimes, cultural connotations trump logic, and AI doesn’t get it. Consider this classic catchphrase from “Saturday Night Live.” One of these words is the punchline. Three are incorrect answers, suggested by Apple’s AI. Do you know the right word?

Why mistakes matter

AI learns by training on human documents, but it can learn human biases — requiring human intervention to correct them. Google Translate used to turn the Turkish phrase “O bir doktor” into “He is a doctor,” though “o” is a gender-neutral pronoun. Why? The algorithm saw more examples of doctors identified as men. After a fix, the AI now offers both a “he” and “she” translation.

How AI can help us

But AI can also help humans function better. People with dyslexia can use AI-assisted tools to make their language expression clearer. The machines learn patterns common to dyslexia in order to predict what people meant to say and substitute the correct words.

Who’s fooling who?

One cutting-edge text generator, called GPT-2, can write full paragraphs and stories after getting a short prompt. Some of the text can be a little wonky, but plenty of it reads as if a human wrote it. Some people are concerned the technology could be used to automate trolling or create fake news.

Your own personal soundtrack

From lullabies to wedding hymns to funeral dirges, music is bound up with emotion. But what makes a song feel right for a moment or mood? Your brain learns, over time, how to pick the perfect song based on your past reactions to music. So does AI.

Are my sad songs sadder than yours?

The human brain is hard-wired to make sense of music — which is why, though some perceptions of music are cultural, others are universal. Researchers asked members of the Mafa, a Cameroon tribe with no exposure to Western culture, to listen to pop songs. Most of the Mafa correctly labeled the songs happy, sad, or fearful, even without understanding the words.

If you’re happy and you know it...

Drag the slider to show where this music falls on a scale from happiest to saddest.

How to measure happiness

You might innately sense that a song is happy, but AI has to be taught. Machines measure mood with a metric called “valence.” The higher the valence, the happier the song. Spotify hires music experts to teach its AI the valence of some songs. That training becomes an algorithm, which the AI then applies to other songs on the platform.

Here’s how your answers compared to Spotify’s valence measurements.

An AI playlist just for you

To build a playlist for you, Spotify’s AI compares music you’ve listened to with songs you might not have heard. (It adds extra rules in hard code, like no Christmas music after December 25.) It also looks at a song’s valence, time signature, key, and “danceability” — a metric based on tempo, rhythm stability, and beat strength.

Do you wanna dance?

Where does this music fall on a scale from least to most danceable?

You got it

Here’s how your answers compared to Spotify’s valence measurements.

There’s wisdom in a crowd

Another Spotify metric compares your listening history to those of other users. If you and a friend have similar tastes, perhaps you’ll like her new favorite song. Another tool trawls the internet for text about songs in news articles and blogs. The platform also considers playlists put together by casual listeners and professionals hired by Spotify.

But what if you hate your friend’s music?

The right data, the wrong song

Even with all this data, the Spotify recommendation engine sometimes offers you a song you hate. That’s because Spotify focuses on technical elements, but doesn’t know the context — a break-up, a bad memory, a pet peeve. Ultimately, taste isn’t perfectly predictable.

An Alzheimer’s playlist?

Music occupies special real estate in your brain: Parts of the auditory cortex respond to music-like sounds, but not to speech. That’s why you might remember song lyrics from decades ago, while forgetting old conversations.

Scientists have found that those special music pathways stay active in patients with dementia. Now, they’re testing how personalized music programs, like the ones generated by Spotify, can help us communicate with Alzheimer’s patients.

Explore other ways our brains — and AI — see the world