Skip to main content
Humans+Robots

Looking for the right makeup? Let AI pick it for you.

Smart hairbrushes, bespoke foundation, and the cosmetics of the future

By Alix Strauss

At Nordstrom’s first flagship store for women and children, on West 57th Street in Manhattan, the beauty offerings are as vast and colorful as you’d expect for a retail mecca: more than 100 brands, displayed over two floors, along with spa and facial treatments. But at the store’s deluxe grand opening last fall, the highlight was four screens — pitching four digital experiences.

There was “The Lipstick Finder,” a TV-sized machine that takes your photo and “applies” up to 400 lip colors, thanks to augmented reality. And “The Fragrance Finder,” a similar machine that delivers an interactive quiz, then narrows down a so-called perfect scent, releasing a sample puff at the press of a button. “The Skincare Finder” snaps a close-up photo of your face, then quickly assesses your skincare needs and makes product recommendations. The “Beauty Stylist Virtual Mirror” helps you virtually “try on” different makeup brands.

Staffers were on hand to help non-tech-savvy consumers — that would be me — figure out how to work these mathematically-focused beauty machines; ones that insist they know what you need, what you’ll like, and what will work best for you. (My mother thinks she knows the same. But I have to admit the computers had a higher chance of success and didn’t fight with me over lipstick shades.)

It’s a sign of how much the multibillion dollar beauty industry is changing. Cosmetics companies like L’Oréal, Procter & Gamble, Neutrogena, and Coty now have in-house tech incubators or departments that focus on technical innovations, incorporating artificial intelligence and augmented reality into a process that, until recently, relied on art as much as science. Want foundation made specifically for your skin type, color, and concerns? Want to monitor the pH level in your skin or the moisture level in your hair? Want custom-printed nail stickers that match the precise shape of your nails? Thanks to technology, that’s all achievable.

“We are always looking for self-improvement, and the idea that a product exists that understands me — and knows what my skin, my hair, and teeth are like — is very seductive,” says Robin Raskin, founder of Living in Digital Times, an events company that brings together technology and lifestyle brands. And consumers, she says, don’t seem to balk at the data collection involved — from everyone who tries out these makeup machines, downloads makeup apps, and opts for a virtual experience.

“Technology can now measure unique aspects of the skin that are impossible to discern with the naked eye,” says Joshua Zeichner, M.D., director of cosmetic and clinical research in the department of dermatology at Mount Sinai Hospital. “The good thing about technology is that they use objective, reproducible algorithms. And the apps won’t lie to you to make you feel better.”


The result is makeup that can be as bespoke as a tailored suit, or maybe more so.

And for someone like me, with difficult skin — hormonal cystic acne sufferers, raise a hand, please — the products can seem like magic. When I tried out Lancôme’s Le Teint Particulier Custom Made Makeup, a trained Lancôme professional used a handheld device, the size of an electric toothbrush, that scanned my skin in three places to detect skin tones and shades.

A device blended the made-to-measure makeup at the counter while I watched — like a paint machine at the hardware store, but for skin.

An algorithm processed my information and fed its findings into an iPad app, then created a match out of 72,000 formula possibilities. A device blended the made-to-measure makeup at the counter while I watched — like a paint machine at the hardware store, but for skin.

Halfway through, the machine took a pause, and the Lancôme rep patch-tested my customized oil-free formulation on my face. The color was a perfect match. The coverage was light and sheer, just as I’d requested. Once finished, the machine popped the foundation into a fancy white container. On the front went a custom sticker with my name, a “Complexion ID,” and an expiration date. And voilà, I had something no one else ever will — unless I have an identical twin no one told me about. (It’s possible. You’ve heard about my mother.)

The author’s custom foundation

Proctor & Gamble offers its own version of personalized foundation: Its Opté Precision Skin System is a handheld wireless device that scans your face with blue LED light, which recognizes age, sun and dark spots, scars, spider veins, tonal imperfections, and even pimples. A facial recognition algorithm and microprocessor analyze your skin, then determine the ideal coverage combination.

Other new technology won’t just match your skin, but monitor it. Last year, La Roche-Posay, another L’Oréal-owned brand, debuted a battery-free, wearable UV-tracking sensor, which measures UV exposure and makes personalized product recommendations. The company is also developing a small, patch-like sensor, placed on the inner arm, that would track your skin’s pH levels by measuring droplets of sweat from your pores — then reveal the results on an app, and, again, suggest products to buy.

Neutrogena has a “360 Skin Scanner,” which uses your smartphone’s camera to measure facial moisture levels, wrinkle depth, pore size, and skin texture. Still in beta-testing is Neutrogena’s MaskiD, a personalized hydrogel sheet mask: A consumer takes a selfie, which goes to the brand’s lab, which sends back a customized mask.

These new “smart products” can also tell you what to do with your hair. L’Oréal’s Kérastase Hair Coach is a “smart hairbrush” embedded with high-tech gadgetry: an accelerometer that measures how fast you comb and a gyroscope that distinguishes how forcefully you brush. The brush can tell if you’re causing damage doing either of these tasks and cross-reference that data with your local weather forecast to suggest what Mother Nature might do to your hair that day. The information goes to an app, which gives a score, advice, tips, and product recommendations.

“We are solving problems in the beauty space that we wouldn’t be able to be solve without technology,” says Guive Balooch, who heads L’Oréal’s technology incubator. While some cosmetics makers, such as Rihanna’s Fenty brand, have famously expanded the number of shades they offer, Balooch says technology can do a better job of making makeup truly universal. “We wanted to solve that issue by creating a machine to measure skin tone, and offer everyone a customized product that they wouldn’t be able to find elsewhere,” he says.


All of this technology turns shopping into a kind of video game, immersive and interactive. Customers are incentivized to download apps and seek out education, which could well lead them toward more beauty products. 

“Mobile devices changed everything. They give you the ‘what if’ before you make a purchase,” Raskin says. “Apps, devices, and machines are giving you makeup counter advice. It’s changed retail. A person behind the counter doesn’t work anymore. Nor does trying on a bunch of products.”

Indeed, several companies are bringing interactivity into shopping; some Sephora stores, for instance, now use “magic mirrors” from ModiFace, a company acquired by L’Oréal, along with virtual try-on lip and lash machines and a fragrance studio with “InstaScent technology.”

The machines offer obvious sanitary benefits — a way to avoid the sharing of germs from the last person who sampled that lipstick. “Trying on products that have been used by hundreds of others is gross,” says Raskin, who curates beauty technology for tech events such as the Consumer Electronics Show. “When you can get the same experience from a machine and not get sick, who wouldn’t want that?”

But Raskin also says consumers may not realize how much personal information these skin- and hair-measuring tools are collecting and storing. Many of these brands, she notes, are looking at detailed data about your body and keeping a record of your purchases. One beauty brand, YouCam, analyzes your wrinkles, spots, blemishes, and dark circles, then keeps track of the stats in a “Skin Diary.”

Balooch says L’Oréal doesn’t store customers’ personal data. “Our approach to data is always keeping it anonymized, which helps us make the algorithms smarter,” he says. “Having anonymized data around UV exposure, skin types, among other information helps us develop better services and clinical studies.”

But even if some companies keep the data, Raskin says, it’s possible that consumers won’t mind — so long as they get the perfect hair or skin care out of the deal.

“We are all guinea pigs in the store or online,” Raskin says. “Brands and devices have become very good at collecting large amounts of data and acting on it. When that data helps you, the customer, it’s OK. To give up some information about yourself to get a better set of choices works for me.”

I’m not so sure it does for me. I still want to think that my physical attributes are private and personal. That my facial analysis at Nordstrom isn’t being used to advance the sales of specific items. And that the photo the machine just took isn’t being used for something else I’m not aware of.

On the other hand, I have that personalized foundation container, with its customized “Complexion ID” and its perfect shade. I know that our data is already monitored every time we shop online or type into a search bar. This is the calculation of modern life, extended to the beauty industry. If algorithms are going to help us look better and younger, they’re going to have to know us — down to our smaller, tighter, and less-noticeable pores.

Published on

Alix Strauss is a writer based in New York.

 

Illustration by Emma Roulette

Humans+Robots

How to prepare for sentient AI

And why the experts hope it never happens

By Schuyler Velasco