Is AI the savior of lipstick shopping as the high street dwindles

In times of economic crisis, sales of cosmetics often rocket. The Lipstick Index, as it’s known, was a phrase coined by Leonard Lauder, chairman of the board at Estee Lauder, in the early noughties when it became clear in times of economic crisis, sales of color cosmetics - in particular lipstick - soar as an affordable way to treat yourself.

But not during the pandemic.

Last year, sales of cosmetics plummeted. With bustling make-up counters in department stores closed for much of 2020, sales of designer brand cosmetics were down by more than 40% according to market research firm NPD, which equates to a loss of £500 million (around $689 million and AU$902 million).

Meanwhile, sales of more affordable cosmetics in supermarkets fell by 22% in the UK, according to the Top Products survey from retail trade magazine, The Grocer, adding a further £183 million loss (around $256 million / AU$218 million).

A combination of the rise of working from home and mandatory masks meant thousands of us, including this writer, ditched our make-up bags altogether. So how can the beauty industry encourage consumers to start buying and wearing make-up again if the aversion to shopping in-store continues for months or even years?

Surprisingly, your smartphone camera and artificial intelligence (AI) could be the answer.

Restrictions leave lasting damage

While lockdown restrictions are lifting, the economic effects of the pandemic have left a lasting effect around the world. Many famous outlets like Debenhams and TopShop have disappeared in the UK, and department stores like John Lewis have confirmed the closure of some locations.

Where other high-street retail has softened the blow of bricks and mortar by adding delivery services, online shopping and cosmetics isn’t as well set up for remote purchasing.

Even if you have a rough idea of the color you want, you’ll no doubt try several testers on the back of your hand until you find the perfect hue for you.

Without the ability to try out a shade, there’s the potential for it to appear completely different than anticipated when applied, and it’s not like color cosmetics can easily be returned to the retailer either if you change your mind.

This could make for an expensive mistake, especially if you consider the price of a designer lipstick can be anywhere from £15 to £50 (around $20 to $70 / AU$25 to AU$90).

So with things looking tricky, a radical solution is needed - and it could come in the shape of technology.

Some brands are using a simple augmented reality (AR) feature, where you can apply different shades virtually to a picture of your face.

Chanel Lipscanner app — (Image credit: Chanel)

Luxury make-up brand Chanel is the latest in a long line of cosmetic brands, which includes Mac, Bobbi Brown, L’Oreal and Maybelline, to launch a virtual try-on app called Lipscanner.

Currently only available for iPhone, this platform uses AR to allow the user to virtually try on any of Chanel’s more than 100 lipsticks and lip glosses, and purchase them online, and it’s the first at-home app that combines virtual try on with a color identifier feature, without the need for extra gadgets (more about this below).

It uses AI that’s been fed thousands of details about lip shapes, size as well as color and texture of cosmetics to identify the color of someone’s lip in an image, (either digital or from a physical magazine page, for example), and suggest the closest shade in Chanel’s range. The more data it accesses over time, the more its accuracy improves.

The algorithm doesn’t just match the shade, it can also identify the finish on the lips and whether the appropriate match should be matte, satin or glossy before placing it on your face to try out.

There are other color identifier apps such as Flawless by Mime, that can select a shade match from a picture you upload, then recommend various make-up products such as nail polish or lipsticks from a variety of brands, but it doesn’t offer the virtual try-on feature.

According to Cédric Begon, Director of the Connected Experience Lab for Chanel, the brand spent almost three years working on Lipscanner, employing an in-house team to build an algorithm that can extract, what Begon calls, “the make-up layer” from a photo.

Begon told TechRadar the algorithm had been fed tens of thousands of images that contained a variety of faces, all with different ages, lip morphologies (shapes), tones and facial expressions, to ensure Lipscanner was capable of recognising colours and offering up Chanel’s equivalent shade.

Yves Saint Laurent Rouge Sur Mesure Powered by Perso — (Image credit: Yves Saint Laurent)

Personalized shades

Some brands such as L’Oreal and Estee Lauder are taking things a step further, and as well as offering apps for virtual try-ons or shade identification, are using AI and AR to create personalized shades for consumers, rather than forcing them to select from predefined colors.

Lancôme’s Le Teint Particulier foundation was launched in 2016 as an in-store service that used AI and AR to identify skin tone and type and create bespoke foundations that perfectly matched an individual’s skin tone.

But now, with millions of potential customers no longer able to had in store (and spending far more time browsing social media) it’s bringing that personalized experience to the home with Yves Saint Laurent Rouge Sur Mesure Powered by Perso - technology developed by L’Oreal itself, which uses AI to blend and dispense enough color cosmetic, such as lipstick or foundation, for one application in the exact shade selected in the app. an exact shade of a color cosmetic, such as lipstick or foundation.

The handy gadget, unveiled at CES this year and going on sale in September, mixes different shades of YSL’s Velvet Cream Matte Finish lipstick range to create the hue desired by the consumer.

The accompanying app offers three ways for the consumer to pick the color they want: selecting a shade from a color wheel, using the identifier to match a color in an image, or getting the app to suggest a color based on your outfit.

Similarly, Estee Lauder’s in-store service - iMatch Virtual Shade Expert - has been tweaked and is coming to users' smart devices.

Estee Lauder has teamed up again with Perfect Core, which developed the technology behind the Shade Expert service to create iMatch Virtual Skin Analysis - an app that can be used at home to identify the most suitable skincare products from the brand for the type of skin the user has.

And it actually works

We tried out Chanel’s Lipscanner app using an image of ourselves wearing lipstick from another brand.

The app was extremely simple to use, and as well as selecting pictures from our camera roll, we were also able to use the phone’s camera to snap an image of a color we wanted to match; a handbag, or dress for example.

The app took a few seconds to analyse the image before offering up a suggestion from Chanel’s range, which we could virtually try on or head straight to the Chanel website to purchase.

We ordered Chanel’s suggested lipstick, and when it arrived the two shades looked very similar side-by-side. We applied the Chanel lipstick on one half of our lips, with the original lipstick from the image on the other.

Again the colors looked extremely similar, although the Chanel lipstick didn’t offer the same color intensity as the original, so we had to apply several coats. The Chanel Lipstick also felt more hydrating on the lips than the original lip color, and had a subtle sheen.

So will AI and AR signal the end of the in-store shopping experience when it comes to cosmetics? Chanel’s Bergon doesn’t believe so. “This is not going to replace human counselling; nothing can replace the trained eye and mind of a well-trained beauty advisor,” he told TechRadar.

June Jensen, Head of UK Beauty, at the NPD Group agrees. “These new advances will help to introduce new customers to prestige beauty while engaging with current customers in a different way,” she said.

“We also expect to see new technology introduced in-store to hyper-personalise the service offered in brick and mortar stores, so that there is a seamless connection between e-commerce and physical store environments.”

So while it seems unlikely we’re all going to pile online to buy our cosmetics, technology is here to stay when it comes to refilling our make-up bag.