AI is a powerful tool that’s becoming an ever more important technology to help beauty brands provide unique user experiences that captivate customers while increasing sales and customer loyalty.
The challenge is: How can beauty brands best incorporate AI technology into the consumer online shopping journey? What AI technologies are emerging, and which are best suited to leverage the power of AI for Beauty Tech.
We will dive into the following topics to give you a holistic view on leveraging AI technology for the beauty sales journey:
- AI Opens the Doorway to Ultra-Personalization
- Using AI Skin Analysis to Gain First-Mover Advantage
- AI Smart Skin Analysis & Skincare Product Recommendations
- AI Beauty Tech: Skin Analysis Real-world Cases
- AI Skin Emulation (Skin Aging Simulation)
- AI Face Attributes Analysis (Face Matrics Analysis)
- AI Foundation Shade Finder & Matcher
- AI 3D Virtual Glasses Try-On
- AI 3D Jewelry Try-On
- Personalization in Beauty: The Right Way to Do It
- Great AI Technology is the Key
Beauty AI Technology Opens the Doorway to Ultra-Personalization
AI was always behind the best augmented reality experiences (AR), the technology that allows users to experience virtual try-on to see exactly how a new hair color will look, different lipstick colors, foundation shades, brow shapes, eye shadow, and other products will actually look on themselves.
In earlier articles, our Beauty Tech: The Complete Guide 2021 and Smart Beauty Mirrors: Virtual Makeup Real-World Applications, we provided an overview of the artificial intelligence (AI) and augmented reality (AR) technologies that are giving customers the ability to look into their mobile devices or computer screens and dive into AI-powered AR virtual try-on experiences.
With companies such as Perfect Corp. leading the way, AI has become so powerful that it is giving companies the ability to offer new experiences to their customers. AI is opening the doorway to what has been termed the ultra-personalization of beauty.
Using AI Skin Analysis to Gain First-Mover Advantage in Skincare
Beauty industry participants who one year ago were taking a quick look at what benefits AI might someday provide for them, are rushing back to kick off projects. They’ve seen that AI, and the AR experiences it powers, is no longer an interesting concept, but absolute table stakes within the industry.
The Beauty industry isn’t alone in waking up to the opportunities:
“AI allows providers of goods and services to quickly and accurately project a 360-degree view of customers in real-time as they interact through online portals and mobile apps, quickly learning how their predictions can fit our wants and needs with ever-increasing accuracy.” - Forbes
Beauty AI technology is driving new opportunities by giving brands the ability to know more about individual users than, in some ways, they even know about themselves. This is because leading edge technology such as Perfect Corp.’s Beauty AI services, can map a user’s face, and evaluate the condition of their skin across a range of metrics. Brands can then use this diagnostic information to make ultra-personal product suggestions to the customer.
Check out how real-time AI Skin Analysis technology works.
We’ll demonstrate how to harnessing the power of AI to provide users with precision skin analysis, which can then be coupled to exacting product recommendations, that users can see for themselves through AI-powered virtual try-on experiences. Beauty Tech is expected to grow to $34 billion by 2024. AI for skincare will be a total game changer.
In this section we are going to explore the latest developments in which consumers are interacting with AI-driven Beauty Tech, including:
- AI Smart Skin Analysis & Skincare Product Recommendations
- AI Age Emulation: Rolling the Years On and Off
- AI Face Chart Mapping: Facial Attributes Analyzer
- AI Foundation Finder & Matcher
- AI 3D Glasses Try-on
- AI 3D Jewelry Try-on
AI Smart Skin Analysis & Skincare Product Recommendations
Dermatologists and other scientists have long used AI technology to help doctors identify skin disorders and symptoms using medical skin scan imaging and tomography. While AI in dermatology is not new, its purpose is largely aimed to assist medical professionals only.
Perfect Corp.’s AI skin analysis technology aims to serve a similar purpose but for a completely different audience. The technology is aimed at providing useful, easy-to-understand skin condition information to the consumer using the widely accessible tool of the camera built into virtually all smartphones.
This is far superior to the online surveys or questionnaires that many skincare brands rely on to get information about users’ skin conditions. Such methodologies are time-consuming for the consumer, inefficient and imprecise. Individuals simply aren’t very good at analyzing their own skin.
Precision AI Out Performs Traditional Skin Surveys
This is why AI skin analysis technology can completely replace the traditional online survey. The technology is able to detect all 8 types of supported skin conditions in 2 seconds, and uses standard measurements to provide unbiased skin condition results. It provides the ultimate convenience to the consumer and the most accurate results to the brands.
While Perfect Corp. technology isn’t intended for use in detecting skin disorders (for medical purposes), it provides a wealth of information, including:
- Helping consumers understand their skin conditions
- Enabling skincare brands to offer products that match specific skin conditions
This is possible because Perfect Corp. skin analysis technology is developed under the highest standards possible, combining skincare professional knowhow, validated by field experts, and using medical grade imaging to inform AI deep learning algorithms.
Perfect Corp. pioneered the use of AI for diagnosing skin conditions, and continues to expand upon this work. Our AI detects more skin health metrics than other comparable services. Its highly accurate AI face mapping and skin health visualization provides instant skin analysis scoring skin parameters including:
- Spots. Dark spots, specifically the “sun spots” or “old age spots,” can appear as a result of natural aging or excessive exposure of sunlight. Perfect Corp. We use a highly trained AI model to identify different levels of dark spots, and effectively exclude non-dark-spot areas like skin mole, facial hair, or other false readings caused by shadows. In addition to providing users with a skin score, we also present the results to consumers in a highly visualized way. Thus, we are using AR technology to display an overlay to highlight areas of concern and use different color codes to show various levels of dark spots (from lighter spots to darker spots) virtually on a user’s facial image.
- Wrinkles. Perfect Corp.’s AI algorithm is trained to identify various types of wrinkles, from fine lines to deep creases. And it focuses on facial areas where the wrinkles commonly appear, such as the forehead, area round eyes, and lower face. We avoid false readings caused by hair strands (such as finagles and bangs), or damaged tissue, such as from scars. We use an AR overlay to clearly mark the different depth of facial wrinkles.
- Moisture. Perfect Corp has developed sophisticated AI deep learning algorithms to analyze and detect skin hydration levels. Normally analyzing skin moisture level is a very challenging task. We have solved the problem by developing robust algorithms to detect various indicators of skin dehydration.
- Redness. Skin redness can be caused by a variety of factors, including excessive sun exposure, over exfoliation, inflammation, and skin dehydration. By the time consumers feel irritation from the condition, it is usually visually apparent. Our AI algorithm for redness is developed to detect subtle symptoms before the condition starts to cause irritation to users. This early detection allows users to apply products like sunscreen or moisturizer to sooth the condition before it worsens.
- Oiliness. Our AI algorithm categorizes the oiliness of skin into three levels, and shows results using our intuitive AR overlays to help users understand specific skin areas requiring more attention. · Texture. Our AI algorithm for texture detects bumps and dents on users’ skin, and gives user an overall assessment of their skin smoothness.
- Acne. Our AI algorithm for acne can identify three types of acne: inflammatory acne, whiteheads, and blackheads. Our distinctive AR overlays mark these problematic areas separately.
- Dark Circles. Our AI algorithm for dark circles detects the under-eyes area for symptoms of dark circles. We are able to clearly mark the size of the dark circle area and use this information to provide users with a metric for the extent of this condition.
- Eye Bags. The same AI algorithm we use for dark circles can also distinguish eye bag conditions. Results are shown with AR overlays to delineate the size and the puffiness of an individual’s eye bag condition.
- Coming Soon. We are currently developing AI algorithms to measure additional skin conditions, including for pores, sagging, and radiance.
Recommended Reading: Coming from our experience partnering with skincare brands, we have another article worth checking out - 'How AI Skin Technology Drives Sales?'.
Top Focus: The User Experience in AI Skin Analysis
Throughout the development of our Beauty AI Technology, we have kept a tight focus on the user experience. This is why we designed our technology to run entirely on the client side. Using an iPhone 8 (or an equivalent Android device), it only takes 2 seconds to show the analyzed results for all skin conditions mentioned above. This makes ours the fastest AI skin analysis solution on the market.
The highly accurate and precise skin advisor, which can be activated on a smartphone, tablet, or accessed via a website, uses our AI-based deep learning, that is verified by dermatologists and skincare experts.
The AI model is based on a skin health database with over 70,000 professionally graded images. And the results were validated through clinical studies of more than 60 patients in multiple test sessions with three different dermatologists. Testing included consistency across different environmental lighting conditions, as well as accuracy compared with professional clinical test equipment. The tool delivers accurate skin assessment for all skin types and ethnicities.
An especially popular feature is that users can track the efficacy of their skincare routine over time and see the results of how products are benefiting them.
All of this has made AI Skin Analysis — and the related product recommendations — popular with Beauty companies and their consumers.
“The artificial intelligence and augmented reality that Perfect Corp. has built into our skin diagnostics tool works so well that it really has an impact on our customers. They see the results as being highly personalized because it isn’t just someone behind a counter suggesting a product. They see the recommendations as being precisely based upon this digital analysis of their own skin.” - Heather Marianna, Director and Co-Founder of Marianna Naturals, and Founder of Beauty Kitchen.
Of course, this boosts sales as well. Marianna notes:
“Since deploying our skin diagnostics tool we’ve seen our sales increase by more than 30%. Just about everyone who scans their skin, buys products. I believe it is the combination of the tool generating actual recommendations based upon the scan, along with the fact they can actually track their progress in enhancing their skin.”
AI Beauty Tech: Skin Analysis Real-world Cases
In the U.S., Marianna Naturals wanted an AI-driven solution to perform skin diagnostics, provide product recommendations, and to allow users to track the progress that their products produced.
“There is a sea of skin-care companies out there, and we wanted to be at the forefront. This meant deploying the highest-quality AI and augmented reality, to stand out from the other companies,” says Joel DeBellefeuille, Co-Founder, President & CEO of Marianna Naturals. “We needed industry-leading AI and AR.”
The company, after surveying what was available, chose Perfect Corp. for the accuracy of its skin diagnostics. Marianna Naturals has found that customers like their products so much—and the skin diagnostics app—that they return to measure improvements from using the products.
Here’s how Heather Marianna, Director and Co-Founder of Marianna Naturals, and Founder of Beauty Kitchen, described their experience with AI skincare:
“As soon as I saw what Perfect Corp. had created for us, I said: ‘Wow!’ The diagnostics were so precise, we saw we could use this on our website, as well as on iPads in our stores.”
The results for Marianna Natural’s use of AI skincare were immediate. In addition to the more than 30% increase in sales, noted earlier, the company enjoyed:
- 300% boost in website traffic
- 330% increase in website dwell time
- Marianna Naturals - Popular plant-based skincare line in the US and Canada
- HOTARU PERSONALIZED - A glowing new skincare brand from Japanese personalization specialist Sparty
Skin Emulation - Rolling the Years On and Off
Critics rave about how Hollywood can deploy millions of dollars in high-tech computational horsepower to add or subtract years to the faces of characters in movies. Think of the AI used to allow Robert DeNiro as both a young man and old in the movie The Irishman, or the reverse aging of Brad Pitt in The Curious Case of Benjamin Button.
With our AI-based emulation technology, you have the equivalent of all that Hollywood computational horsepower in your smartphone. AI Facial Aging Simulation through our cutting-edge AI Generative Adversarial Network (AI GAN) technology is used to generate facial aging and anti-aging results through facial simulations, allowing consumers to visualize the effects of products with their own faces.
And users love it. We are all curious about how we might look 20, 30, or 50 years from now. And it is fun to roll the years backwards to get a glimpse of your younger self.
From a Beauty Tech perspective, the AI behind skin emulation allows companies to give their consumers a look into the future, and suggest products that can help slow the aging process.
Face Chart Mapping: AI Facial Attributes Analyzer
To fully appreciate what AI is bringing to Beauty Tech, it is helpful to take a look at the AI Face Attribute technology that we bring to further complement virtual try-on experiences, an area in which Perfect Corp. has long been the leader in this exacting technology.
Perfect Corp. AI Face Attribute is a powerful technology designed to help beauty brands learn about their customers’ facial characteristics in the most convenient way. To provide truly personalized consumer experience, beauty brands need to give unique product recommendations based on an individual’s differences.
Traditionally, it’s been utterly challenging to get accurate information about a user’s facial structure and characteristics. You can’t expect a consumer to be able to accurately decide whether their face has diamond shape, heart shape, oval shape, rectangular shape, round shape, square shape, and so on - let alone use this information to enable useful makeup advice. In many scenarios, it’s not even possible to use the traditional online questionnaire to the answers because consumers simply have no idea which categories they belong to.
Our technology creates a seamless bridge between the gaps. Our AI deep learning algorithm detects 70+ types of facial attributes for distinctive face shapes, eyebrow shapes, eyes shapes, cheeks shapes, nose shapes, and lips shapes.
Only with this level of details, can beauty brands create specific and personalized recommendations back to the individual users.
Our AI Face Attributes Technology, combined with our AI deep learning, is behind our ultra-lifelike virtual try-on experiences for hair, brows, lips, foundation, and other cosmetics. It also provides the basis for our skin diagnostics and product recommendation capabilities. Together, these provide the Beauty Tech that companies need to attract, engage, and captivate their customers.
Partner Success Use Case
In Japan, Kanebo Cosmetics’ makeup brand Coffret D’or used Perfect Corp. AI skincare technology to launch its digital beauty service COFFmi. The application, which allows customers to experience personalized makeup consultation through the LINE App, is based on Perfect Corp.’s AI Face Attribute solution, which accurately analyzes a user’s facial features in seconds.
When the company tested user engagement for its personalized makeup page against another promotional page on its website without the virtual makeup feature, they found:
- 11x increase in page views
- 248% increase in average time spent on the page
Visit here to learn more about Coffret D’or and its use of Perfect Corp. AI Face Attribute.
These impressive figures are in line with what nearly all of our customers find. It reflects a genuine hunger on the part of consumers for the chance to take an AI-powered journey into the world of AR. As the growing power of AI allows for ever more personalized experiences, the consumer demand for such experiences will only grow.
Foundation Shade Finder, Powered by AI
The personalizing power of AI is taking the guess work out of finding the best foundation shade. Our AI-powered Foundation Finder provides precisely accurate foundation match through smart detection of up to 89,969 shades with unlimited grades from light to dark, as well as unlimited undertones from warm to cool. Highly accurate AR facial detection and machine learning technologies detects and matches customer skin tone.
Estée Lauder created its popular iMatch™ Virtual Shade Expert using our AI technology to create a YouCam Makeup virtual try-on-color match experience. Personal product recommendations using skin tone allows the customer to choose their perfect shade or other closely matched shades. After matching, customers are provided product information to purchase their personal favorite foundation shade.
Watch the AI Foundation Finder & Matcher technology in action.
“Online customers who use the tool have 2.5 times higher conversion rates. It’s not about just technology for the sake of technology. What we are developing for the Estée Lauder brand is technology to enhance the consumer experience. Today, we are leveraging tools to really make sure we can engage with the consumer in an even more meaningful way around the world.” - Stephane de La Faverie, Global Brand President, Estée LauderRead the full stories about our other success brand partners:
- Tarte Cosmetics - US based makeup, skincare & beauty brand
AI 3D Virtual Glasses Try-on
When Vogue, as noted earlier, wrote “What if every time you shopped online you could see yourself in the clothes? Literally,” they did a good job of describing how AI can now be used to let you try on glasses and jewelry—virtually.
Check out the virtual glasses try-on experience in real-time.
The same Perfect Corp. AgileFace technology that precisely measures and maps the face for skin diagnostics and virtual try-on of cosmetics, can be used to virtually try on glasses and jewelry.
With our AI 3D virtual try-on for eyeglasses, trying on glasses has never been easier, or more efficient. And instead of an assistant holding a ruler up to your face to measure the distance between your eyes, a range of more precise measurements are generated by the application—helping to ensure a perfect fit.
Making it Easy for Brands: Auto 3D Modeling for Virtual Glasses SKUs
As with so much else in the Beauty industry, the COVID-19 pandemic has brought what is likely to be a long-lasting resistance to trying on anything touched by others—including glasses. This underscores the need for virtual try-on. This in turn means enabling virtual try-on across a brand’s complete set of SKUs.
To solve the pain point of setting up virtual glasses SKUs, we put ourselves into the designer brands position and created a streamline workflow to help the brands to quickly and effortlessly create a pool of 3D glasses models precisely representing the real products. We’ve essentially brought automation to creating the 3D model for glasses. Using our AI technology, brands only need to upload 3 images of the glasses to create a full 3D model.
AI 3D Virtual Jewelry Try-On
For jewelry, our AI 3D Jewelry Try-On is so precise that if a user selects earrings, and turns their head, the earrings may swing, depending upon their shape, size, and the momentum of the head turn. This is just one example of the physics and other attributes we’ve brought into our AI 3D Jewelry Try-On.
Of course, as in so many AI applications, there is an enormous fun factor. Just how would I look wearing that $275,000 diamond necklace?
It’s like having your very own highly personal shopper—residing within your smartphone.
The precision of the virtual try-on experience, along with the fun factor, help create AI-driven experiences that support even more personalized customer journeys.
Personalization in Beauty: The Right Way to Do It.
As brands embrace Beauty AI Technology to provide enriching experiences for their customers, here are some basic guidelines to maximize success:
Tip #1 - Personalization. The power of AI should be used to unlock the door to personalized experiences. Avoid creating the impression that a user is just being served up a random selection of recommendations. Make it specific to their needs.
Tip #2 - Explain Why a Product is Recommended. Personalization is enhanced when you let a user know why a recommendation is being made. Does it have to do with the structure of their face, the color of their eyes, length of their hair, shape of their lips? The more personalized you can make it, the more entranced your customer will be. Let her know this recommendation is just for her.
Tip #3 - Keep it Fun and Easy. The virtual try-on experience must be streamlined and easy. The easier you can make the experience, the bigger the WOW factor you can create. You want the technology to be working so efficiently behind the scenes, that all they see is a magic mirror that lets them fly through a whole world of beauty options.
Tip #4 - Educate. Beauty AI Technology is still new and emerging. Many consumers may be experiencing it for their first time. They may appreciate the experience—and trust the recommendations—even more if you briefly describe that AI and AR are analyzing their attributes and making recommendations based on that.
Tip #5 - Ensure Consistency. Consistency of results is important. Ensure the choice of your AI-powered technology can produce consistent and repeated results on the same individual. Consumer lose trust in technologies quickly if they see random results between repeated usages.To read more about best practices in AI beauty tech:
- Top UX Tips to Create the Best Web Virtual Try-On Experience
- 6 Things to Consider When Choosing Beauty Tech for Your Online Store
Great AI Technology is the Key
Great AI technology is the key to opening the door to this new world of ultra-personalization. Some of the first AR experiences enabled consumers to experiment with new hair colors—through VR. Though several players entered the field of offering virtual try-on for hair, the results could be very disappointing. Hair coloring could flow onto a sweater, or when the person turned their head, the color was left on the wall behind them.
Perfect Corp. avoided such problems from the start because its AR has always been powered by exceptional AI. Our AI identified hair on a strand-by-strand level, which enables captivating realism for virtual try-on for hair colors—and more. Requiring even more precise AI is rendering brows and lashes. Our AI, again, enables completely lifelike AR experiences, something our customers were quick to notice, and appreciate:
- “I was skeptical when we gave the challenge to Perfect Corp. I never thought they could capture the multidimensionality [of hair] and the subtle differences between those five shades of brown. But they did, and our whole team was amazed. … And they have been amazing us ever since.” - Tyler Wozny, Senior Product Manager, Madison Reed
- “Realism was an absolute essential for us, and realism is difficult to achieve with something as subtle as hair coloring. … The thing that really stood out about Perfect Corp. for us is their use of artificial intelligence. Their AI algorithms work with their augmented reality that gives it that very realistic appearance. In contrast, other try-on tools in the market are just straight-up augmented reality, and not as realistic-looking.” - Heather Lockwood, Director, Digital Transformation and Strategy, Aveda
- “We are averaging more than 10 million try-ons a month with Ardell Lashes. And we are delighted that a large percentage of those are coming from our international markets. For a brand like Ardell, that is distributed in over 100 countries around the world, it’s hard to think of another tool that could allow us to provide such a consistent consumer experience on a global scale.” - Marwan Zreik, Vice President, American International Industries
This same AI technology is required to place color precisely on lips, around eyes, and on the skin for foundation. Extending the power of AI from hair to skin required even greater technology, and the AI we’ve developed to do this is providing industry-moving rewards.
AI is the Biggest Trend in Beauty Tech
Artificial Intelligence (AI) is everywhere—from the emergence of self-driving cars, to massive trading strategies on Wall Street. AI is also powering a new world of ultra-personalization in Beauty Tech.
The results will revolutionize the way customers interact with beauty products, and the world is taking notice:
“Perfect Corp.’s award-winning Beauty AI solutions have transformed the beauty shopping experience online and offline through interactive, hyper-realistic virtual beauty try-ons, and instant skin and hair diagnostics that change the way consumers shop and connect with brands. A ‘one-size-fits-all’ approach is no longer appealing to consumers, especially when it comes to beauty products, and AI has allowed the level of personalization that has never seen before.” - Analytics Insight
“There are now multiple AR and artificial intelligence (AI)-powered apps that allow users to test out makeup products, skin-care products, and even nail polish with an easy click of a button, right in the palm of your hand. … The growth of AR and AI technology during the pandemic has been so successful, in fact, that makeup apps like YouCam are using their technology to help keep other beauty businesses afloat.” - Allure
“Welcome to the beauty industry’s golden age of artificial intelligence. What once was considered a novel, even borderline science-fiction idea is now becoming much more accessible to the masses thanks to new innovations across nearly every product category.” - Coveteur
“Perhaps the most compelling reason that digital models may become the standard is that they are the ultimate symbol of individuality and inclusivity. What if every time you shopped online you could see yourself in the clothes? Literally. If the algorithms were fed enough data about us as individuals they could generate content featuring us, each one of us, as models.” - Vogue
While there has long been excitement about AI in the beauty industry, Perfect Corp. has been an innovation pioneer in this field, developing the AI algorithms, deep learning, and precision AR to give brands the technology they need to captivate customers and increase sales.
While we are all hoping that the COVID-19 pandemic will soon be history, behavioral changes that it induced are likely to be long lived—especially if variants emerge that enable the problem to linger. Those who lived through lock-downs won’t be eager to stick a finger into samplers, and there may be long-term reluctance to return to crowded stores.
The great news for the Beauty Industry is that the AI and AR that enabled their customers to experience the joys of virtual try-on have relevance far beyond the need to isolate.
The pandemic opened the door to a new world of Beauty AI Technology that gives consumers a hyper-personal environment in which to explore. This same environment gives brands the opportunity to forge ever tighter bonds with their customers. AI skincare provides a metrics-based analysis of skin that customers can trust and engage with. Brands can use this technology to match skin condition to product recommendations, and customers can see for themselves how well the products work for them.
Again, a door has been opened to a new world—one that customers love, and that brands can use to fine-tune personalized offerings. We expect that such ultra-personalization is just the beginning of an entirely new realm within Beauty Tech.
Get expert advice from Perfect Corp. and be part of the beauty tech trend. Contact us for details on beauty tech solutions trusted by 300+ brands globally.
Read our “Beauty Tech: The Complete Guide 2021” to unlock powerful ROI secrets.