(Updated on January 28th, 2022)
As the world emerges from the pandemic, the beauty industry is experiencing a digital transformation. More than ever before, consumers are turning to brands for digital services and experiences that offer personalized product advice and inspiration. In order to effectively engage consumers and provide impactful shopping experiences, it has become essential for brands to create digital experiences that entertain shoppers and help each customer find their perfect product match. Beauty Tech, is an emerging new tech trend encompassing AI and AR-powered virtual try-on technologies and diagnostics. Beauty tech has already helped hundreds of beauty brands grow their sales figures and increase customer engagement.
In “Beauty Tech: The Complete Guide 2022,” we will show you how some of the world’s most iconic beauty brands have leveraged beauty technology and virtual try-on experiences to enhance their omni-channel shopping journey, in many cases driving as high as 2.5x increases in sales conversion, as well as triple-digit increases in customer engagement and digital engagement metrics such as time spent on site and product add-to-carts. We will also provide a complete guide to getting started with beauty tech, from how to select the best solution for your brand to how to integrate the technology across your brand’s ecommerce and retail shopping touch points. Whether you are a large beauty company or a small indie brand, Perfect Corp. can help you transform your customer experience with the power of beauty tech. Keep reading to learn:
Recommended Reading: If you already read this article, we invite you to check out 'Smart Beauty Mirrors: Virtual Makeup Real-World Applications.'
Tech Enters the World of Beauty
Cosmetics and humans have a very long history. Egyptians, 7,000 years ago, saw beauty as a sign of holiness—and used copper carbonate to create green eyeshade. To the Greek philosopher Plato, beauty was indistinguishable from philosophy. Ancient China pioneered painting fingernails. Tribes in Africa—as elsewhere—have used face paints and other elements of cosmetics to denote affiliation, roles, and status. Across the years, poets, painters, and philosophers have weighed in on the role of beauty in cultures around the world. And virtually everywhere, cosmetics in one form or another—for men and women—have played a role.
A few years ago, ELLE magazine, in a photo essay “21 Portraits of Beauty Around the World” provided a nice definition by the Romanian photographer Mihaela Noroc, who said: “Beauty is everywhere. … We are all beautiful because we are different." It’s only natural that the world of high technology would soon find a role in beauty — especially with tech’s innovations in artificial intelligence (AI) and augmented reality (AR). These technologies are helping to democratize beauty and help each person find the best product for their unique features.
What is Beauty Tech?
One could trace the birth of Beauty Tech back at least to early paint and photoshopping applications on desktop computers, though it really took off with the more recent advent of AI and AR technology.
“Although beauty tech dates back to the days of boxed computer programs (Cosmopolitan magazine put its name on “virtual makeover software” all the way back in 1998), it’s only recently that big brands have started to invest heavily in the space. The surge in interest has been driven in large part by advances in smartphone technology and the development of advanced beauty tech solutions such as Perfect Corp.’s AR virtual try-on solution for makeup and AI skin diagnostic technologies for skincare brands. Consumers now have their smartphone with them at all times, and through the power of beauty tech, can snap a photo to try on makeup shades and receive a personalized beauty recommendations in seconds.
Using Beauty Tech to Create Personalized Shopping Experiences
Today, beauty is not one size fits all, and when consumers shop for new makeup products, they are seeking out personalized advice and recommendations from brands. The power of AR and AI technology for beauty brands lies with its ability to cater to individual consumers and their specific beauty needs. The deep learning algorithms that drive AR and AI beauty tech help brands provide mass scale personalization to consumers. With these technologies, every customer can try-on products virtually and receive a personalized product regimen.
According to a recent study by Forrester, 77% of consumers have chosen, recommended, or paid more for a brand that provides a personalized service or experience. Forbes also recently reported that 71% of consumers feel frustrated when shopping experiences are not personalized. From this data, it is clear that beauty tech tools and personalization strategies are becoming essential to create strong customer experiences and shopping journeys.
Why is Beauty Tech important?
“AI and AR technology is fast becoming a crucial part of a beauty brand’s digital transformation strategy, as more shoppers become accustomed to shopping online and seek out more opportunities for personalization,” says Perfect Corp. founder and CEO Alice Chang, “Virtual try-on and personalized product recommendations not only help to boost consumer confidence when making purchase decisions, but also give brands and retailers a chance to reimagine their consumer journey.”
Entertaining Shoppers with Virtual Try-On and Beauty Tech
There is something inherently fun about AR and virtual try-on. This is because AR, by its nature, combines the real world with an augmented realm in which the user has control far beyond what might be possible—or at least practical—in the real world.
Fun factor example. Madison Reed finds that customers enjoy the fun factor of exploring hair color options.
The sheer fun of AR captivated the world a few years ago with the Pokémon Go craze, which blended game objects with the real-world environment a person happened to be strolling through. Suddenly parks were jam-packed with people staring at their smartphones, delighted to see a colorful character suddenly sitting in a real tree growing out of the ground. A headline in the New York Times captured it succinctly: “Pokémon Go Brings Augmented Reality to a Mass Audience.
Within the beauty industry, beauty tech and AR-powered virtual try-on experiences help captivate audiences and shoppers in a similar way. These technologies add a layer of interactivity to the traditional beauty shopping experience. Users can digitally explore dozens of makeup shades and see these products come to life through virtual try-on. This guarantees a personalized shopping experience that is also fun and entertaining for the shopper.
“We hear from customers who have been going to the salon for 40 or 50 years, in some cases, and have never colored their hair at home until trying us,” says Tyler Wozny, Senior Vice President, Digital Product at Madison Reed. Madison Reed recently launched a hair color virtual try-on experience using Perfect Corp.’s AR technology. The experience allows users to virtually try on more than 50 Madison Reed hair colors, helping each customer to find the perfect shade and feel confident about their next hair color transformation. “There’s a sense of pride that’s wonderful to see. Customers tell us the whole experience is great from start to finish. And again, our Perfect Corp. AR tool plays a big role in that.” Madison Reed has experienced a 38% increase in conversion after implementing the virtual try-on tool.
AR technologies such as the virtual try-on solutions provided by Perfect Corp. are becoming essential to entertain and engage beauty consumers along the shopping journey. Consumers are turning to digital channels for beauty product advice and inspiration more than ever before. In 2021 alone, there have been billions of virtual makeup looks tried-on digitally in Perfect Corp.’s YouCam Makeup app. As the world emerges from the pandemic, try-before-you-buy experiences are integral for creating personalized and engaging shopping experiences that drive purchasing confidence. AR virtual try-on technology has undergone major advancements over the years and now allows beauty brands to enable interactive and hyper-realistic digital experiences across the entire range of color cosmetics products, including lipstick, eye shadow, eyebrow, eyeliner, mascara, blush, highlighter, and foundation.
The hyper-realistic virtual try-on technology even extends to such categories as skincare, hair color, glasses, jewelry, earrings, nail polish, watches, and more, effectively transforming the shoppers’ beauty journey and empowering consumers to make informed purchase decisions.
Brand's Success with Beauty Tech
In addition to the fun and entertaining aspects of augmented reality—and perhaps in part because of the fun aspects—technologically advanced AI and AR solutions deliver tangible, measurable results to beauty brands, and help retailers drive sales.
The business value of Beauty Tech is real and cannot be overlooked. In fact, the real-world results often surpass expectations, producing exceptional impacts on business across a spectrum of scenarios, both online and offline.
Here’s a look at how some digital-savvy companies have experienced significant uplifts from various applications of beauty technologies:
- Estée Lauder. Estee Lauder saw 2.5 times higher conversion rate with Perfect Corp.-powered Lip Virtual Try-on and found increased customer loyalty from Perfect Corp.’s AI-driven iMatch™ Virtual Shade Expert.
- M·A·C Cosmetics experienced a 200% increase in customer engagement within the first month of implementing digital solutions and virtual try-on.
- Jane Iredale Cosmetics experienced a 300% increase in average website session time after integrating virtual try-on technology. The brand also found that virtual try-on more than doubled conversion as consumers interacting with VTO converted a 117% higher rate.
- Decorte saw an 85% increase in sales after implementing virtual try-on.
- e.l.f Cosmetics. e.l.f Cosmetics saw a 200% higher conversion rate for online consumers using Perfect Corp.-powered virtual try-on.
- Benefit Cosmetics. Benefit Cosmetics enjoyed both a 101% increase in time-on-site and a 20% increase in add-to-cart using its browser-based customer AR experience powered by Perfect Corp.’s YouCam virtual try-on solution.
- Tarte. tarte saw 200% increase in sales for customers experiencing virtual try-on, triple-digit increase in time spent on site, 30% jump in add-to-cart using virtual try-on.
- Marianna Naturals. Skincare brand Marianna Natruals boost website traffic by 300%, dwell time by 330%, and sales by 30%.
- Mineral Fusion. Sees 40% boost in conversion & 141% sales increase after deploying virtual try-on.
- Madison Reed found a 38% improvement in conversion with hair color virtual try-on and 10X rise in hair color sales, using Perfect Corp. AI-powered AR technology.
- Aveda. Aveda saw a 220% jump in Try-On Tool traffic and a 5X boost in the use of Aveda’s Salon Locator vs. site average, and 14% higher sales from customers who engaged with their Perfect Corp.-powered Virtual Try-On.
- Meredith found a 300% boost in engagements with digital ads that offered their Perfect Corp.-powered virtual try-on experience.
These impactful results illustrate why Beauty Tech has become essential for beauty brands to drive customer engagement and sales. It is an immensely powerful tool that not only helps beauty brands deliver personalized product recommendations to customers but also provides real business value.
How Beauty Tech Can Improve Ecommerce KPI Metrics
There is a rapidly growing trend and desire to engage with customers digitally. As more brands start to offer digital experiences to enhance the shopping journey, innovative beauty technology will become essential to meet consumer expectations. Increasing sales through the use of Beauty Tech will always be the ultimate goal for brands, however, there are also other important digital metrics (KPIs) that can be improved as a result of implementing AI, AR and beauty technologies.
Important digital metrics that can be significantly improved with beauty tech experiences include:
Basket Size, CR (Conversion Rate), and AOV (Average Order Value)Neilsen reported 51% of consumers are willing to use AR to assess products, as the “try-before-you-by” experience is now helping to address real consumer pain points and help customers make more confident purchasing decisions. In fact, e-commerce platform Shopify reported that interactions with products featuring AR content and filters showed a 94% higher conversion rate than products without AR. With technology such as virtual try-on, customers can try-on and discover more products than they would during the traditional in-store product try-on experience. For this reason, they are more likely to find more products that they love and purchase more. As a result, basket Size, CR, and AOV are three important metrics that brands can see a significant improvement with after implementing beauty tech.
Digital EngagementsBeauty Tech delivers useful and interactive content that helps customers find the products that are best fit for their individual needs. AR beauty filters are fun and engaging, bringing products to life before the customer’s eyes. As a result, consumers want to engage with such content more and more, and will often stay on a brand’s website longer trying on multiple shades and products. This higher engagement leads to better click-through-rate (CTR), and thus more sales.
Time-on-SiteWhen beauty shoppers can try on products virtually, they stay on a brand’s website longer. Beauty Tech adds interactive elements to the consumer journey, making the entire experience more entertaining, which in turn increases the average time they spend on a brand website. KAO’s makeup brand KATE reported that users stayed 4.3 times longer on the site that offered a virtual try-on experience compared with the previous campaign site without virtual try-on features.
New & Returning TrafficBeauty Tech helps brands and retailers build a vital emotional connection with consumers, which leads to higher brand awareness and loyalty. Customers tend to return to brands they have positive associations with, and Beauty Tech delivers immersive and interactive shopping experiences that create such associations. Moreover, when customers are satisfied with their online shopping experience, chances are that they would come back for more or recommend the brand to their friends. This brand loyalty leads to higher return traffic numbers. Additionally, as many consumers are now searching for ways to virtually try-on products, brands are also likely to see their new site traffic increase as more and more consumers discover their virtual try-on tools through organic search.
Product Return RateBeauty Tech significantly increases confidence in purchase decisions—effectively lowering return rates. With virtual try-on, customers can directly see how the product looks for them, boosting confidence in making purchases. Beauty consumers who can experience the product virtually tend to avoid common shopping mistakes, leading to lower return rates.
Beauty Tech’s Growth and Evolution
Beauty Tech — especially the realms of AI-powered AR — has proven to be so helpful, and captivating, that it has enjoyed rapid growth and evolution. Moreover, recently the expedited shift to digital beauty behaviors due to the global pandemic, as well as demand for hyper-personalization and lack of physical product samples/in-store makeup & skincare services has led to an increase in virtual try-on usage and beauty tech’s growth. Along the way, Beauty Tech has found some fascinating uses—some expected, others emerging as surprises. Examples include:
- The Birth of Virtual Try-On Communities
- Creating digital beauty consultations and AR-enabled video tutorials
- Completely virtual personalized makeup tutorials
- Providing Touchless Try-Ons During the Pandemic
- Doing the Impossible “30 Shades in 30 Seconds”
- AI-powered skin analysis and simulated results for skincare products
The Birth of Virtual Try-On Communities
The joys of AR-enabled virtual try-on has given birth to a vast number of online communities in which users share their looks and see what others have done. Vogue refers to the rapid growth of virtual try-on communities as the democratization of beauty. “Social media, particularly Instagram and YouTube, has been widely accredited with the democratization of beauty,” Vogue writes. “The authority once attributed to European fashion houses now lies at the hands of online communities, helmed by the likes of Charles Jeffrey, Jackie Aina, and Nikkie de Jager. In 2018, beauty content generated more than 169 billion views on YouTube. The result is a highly engaged audience, fluent in small-print formulas and who know the difference between a retinol and retinoids.”
These communities offer brands undreamed of opportunities for directly interacting with their customers, gauging which products are most tried, engaging in real-time market testing, and gaining insight into where trends are going. Harnessing all of this requires the use of an AR platform that supports analytics and can be seamlessly integrated across the 360 omnichannel shopping journey. Used optimally, such platforms can greatly enhance customer loyalty, as well as on-site dwell time and larger cart purchases. Perfect Corp. is the only beauty tech solution provider to offer 360 omni-channel digital solutions that engage consumers, provide personalization, and integrate seamlessly across platforms to ensure unified and comprehensive data insights.
Beauty Tech Enables Product Discovery and Education
AR virtual try-on experience on a beauty eCommerce site.
Well-crafted AI + AR beauty tech can help consumers find the best products, but it can also educate consumers on these products while helping them understand how to use products to create the best results.
Many consumers turn to YouTube videos, for product advice, but makeup videos can often lack the interactivity and personalization provided through AR. With beauty tech solutions such as Perfect Corp.’s YouCam Tutorial makeup teaching platform and Beauty Advisor 1-on-1, consumers can access personalized advice and education from the comfort of their homes, in a fully digital setting. AR can help consumers understand how to best use products for their individual features and beauty needs.
With Beauty Advisor 1-on-1, consumers can connect with beauty advisors through a real-time video call. Customers can chat with the advisor, receive beauty recommendations, and even try-on recommended products through AR-powered virtual try-on.
With YouCam Tutorial, a groundbreaking AR-powered makeup learning platform, consumers access personalized, guided makeup tutorials. The steps of each look are mapped out on a customer’s face in real-time through a brush stroke overlay animation. Each highly realistic and inclusive virtual makeup tutorial is like a personal virtual makeup coach, allowing customers to visualize the step-by-step makeup application process on their own faces.
With these tools, customers can not only try-on and discover the best products, they can also access in-depth product education that empowers them to create the best beauty results. This type of personalized consultation and education was once only available to consumers when visiting a store, but it is now available digitally through the power of beauty tech.
Beauty Tech can bring awareness, provide education, show hyper-realistic results, and help users make better, more confident purchasing decisions.
For the user, beauty tech and virtual try-on can simplify the beauty shopping experience and help customers find the perfect products.
Providing Touchless Beauty Tech as the World Emerges for the Pandemic
The COVID-19 pandemic has likely spelled the end for shared makeup samples. Allure puts it this way: “The last time you dabbed a free spooley into a tube of tester mascara may have been the last time. … But even before the storefronts shuttered, augmented reality technology was on the rise in beauty. AR allows for virtual try-on, in which a customer can swipe on a new lipstick shade, hair color, or brow shape via a desktop browser or mobile app.”
AR-powered virtual try-ons eliminate the concern of contaminated samplers—or even needing to venture out of the house to visit a store. As consumers become more conscious about health, virtual try-on has emerged as a hygienic and safe alternative to traditional beauty product testing experiences.
“We feel we are in a powerful position because we already have a touch-free way for our customers to explore our products,” says Emily Dybwad, Director, Global Digital Marketing at Benefit Cosmetics. “Even before COVID-19, customers were reluctant to use samplers. We are now in an environment when consumers will likely be even more vigilant. All of this makes our Brow Try-On an even more welcoming experience for our customers.”
During the pandemic, Perfect Corp. further optimized its’ technologies to meet the needs of post-Covid beauty consumers. In order to create a safe virtual try-on experience for consumers, Perfect Corp. introduced gesture control and voice activation for in-store virtual try-on integrations. With gesture control, customers can use hand gestures to navigate the virtual try-on experience without ever touching the screen. With voice activation, customers can select different products or categories by saying corresponding words and numbers. Perfect Corp. also introduced a virtual lipstick arm swatch feature which allows consumers to try on multiple lipstick shades and textures with a digital swatch display on their forearm, all while safely wearing a face mask.
In addition to these optimizations, face mask detection has also been added to in-store virtual try-on experiences to comply with face mask requirements. With this enhancement, the AR virtual makeup try-on technology can accurately detect whether consumers are wearing masks and apply virtual makeup and looks around the mask area.
Creating Sustainable Beauty Shopping Experiences
AR and AI-powered virtual try-on technology is also driving sustainability in the beauty industry by creating a fully digital product try-on experience for makeup and beauty brands. In the past, a customer who was interested in buying a new beauty product would have visited a store, opened up a product tester, and tested the shade on their skin. With the beauty industry being one of the biggest contributors to plastic and packaging waste, this traditional product testing environment is not eco-friendly. With Perfect Corp.’s virtual try-on technology, beauty brands can create a fully digital product testing environment. With this technology, consumers can digitally explore products and try-on hundreds of shades virtually without ever touching or opening a physical product. Whether in store via a smart mirror or on a brand website, this technology can seamlessly be integrated across the omni-channel shopping journey to minimize product waste and provide an engaging product try-on experience. These virtual try-on technologies help brands reduce their carbon footprint and prioritize their Environmental, Social, and Governance (ESG) -focused strategies by reimagining the consumer experience through a more sustainable and ethically-conscious lens.
Doing the Impossible “30 Shades in 30 Seconds”
One of the most remarkable and groundbreaking aspects of AR is that it allows users to try on dozens of products in seconds, an experience that would be impossible during the traditional product testing experience. Estée Lauder’s first implementation of Perfect Corp.’s virtual try-on solutions offered customers the magic of being able to try “30 Lipstick Shades in 30 Seconds.” Presently, lipstick aficionados can virtually try on every shade in Estée Lauder’s Pure Color Envy lipstick range with the brand’s Virtual Artist Tool. In real life, trying 30 shades in even 30 minutes would be unlikely, as the lips would become too tender from the cycle of applying and then removing lipstick so many times. The same is true for just about any cosmetic—and even more so with something as time-consuming and as difficult to reverse as dying your hair.
“Customers see exactly how a color will look on them.” says Marwan Zreik, Vice President at American International Industries, “They can instantly change their hair color from pink to blue to green to red to purple. This allows them to choose the perfect color for their own personal style.”
AI-powered Beauty Tech Enables Personalized Skin Analysis
Beauty Tech has advanced and evolved beyond just the virtual makeup try-on experience, and Perfect Corp. has now developed tech solutions across makeup, men’s grooming as well as skincare. By incorporating Artificial Intelligence (AI) technology, Beauty Tech has become smarter and more advanced. Using machine learning and deep learning algorithms, Beauty Tech can now provide skincare customers with product recommendations personalized to their individual skin types and concerns.
Recommended Reading: we have a whole article showing everything you'll need to know about AI skin analysis technology - AI Skin Technology: The Complete Guide 2022
High-definition cameras combined with AI-powered processing allow consumers to use their own smartphone as a skin diagnostic tool accessible anytime, anywhere. A typical use case is offering an online ‘skin doctor’ experience—accessible 24/7— that can assess one’s skin concerns and grade them professionally in terms of severity in real-time. The AR technology can then create multiple layers of overlay images to show the exact location of wrinkles, fine lines, spots, uneven texture, and acne. For the very first time, consumers can receive a precise objective evaluation of their skin condition without visiting the dermatologist. Brands can integrate these tools across their digital channels to provide customers with a skin assessment and customized product regimen based on those results.
Similar technologies have also been deployed within retail environments. The advancement of AI Skincare over the past 12 months has been tremendous. With Perfect Corp.’s AI Skin Diagnostic Technology, consumers can obtain a personal skin analysis across 14 skin concerns, including:
- Dark Circles
- Eye bags
- Skin Firmness
- Droopy Upper Eyelid
- Droopy Lower Eyelid
- Visible Pores
With AI skin analysis, it is extremely important to ensure that the diagnostic results are verified by dermatologists and skincare experts. Thus, Perfect Corp. also collaborated with experts in the field to ensure the AI skincare technology can provide accurate, meaningful, and validated results to skincare shoppers.
Thanks to collaborative partnerships with skincare brands like Marianna Naturals, Perfect Corp.’s AI skincare service can be trusted by brands and consumers alike. Since launching is AI Skin Analysis experience, the brand has seen a +300% increase in time spent on site and a +30% increase in sales.
Beauty Tech Essentials:
Before taking a closer look at each of the elements that go into Beauty Tech, it’s helpful to get an overview of how it all works together. According to EOS Intelligence, “The use of AI, augmented reality (AR), virtual reality (VR) as well as complex beauty devices has revolutionized the way consumers perceive, apply, and select beauty products. Moreover, in the age of online retail, it enables companies to maintain a similar personalized level of service that would otherwise require a physical interaction with a beauty consultant.” Once a futuristic gimmick, AR & AI beauty tech has become an integral part of the beauty shopping experience for makeup, skincare, nail polish, men’s grooming, and hair brands of all sizes.
Beauty Tech represents a merging of hardware and software. Some companies are creating specific-use hardware devices — such as for analyzing current skin conditions to provide more precise product usage or using handheld inkjet printers to hide blemishes or create masks.
However, the most common hardware element is the ubiquitous smartphone. These days everyone has one. Model upgrade by model upgrade, their cameras become more precise and their operating systems, and portfolio of apps, become more powerful. Add to that the increasing speed of mobile internet connectivity, and you have a powerful environment in which AR and AI can flourish.
Ideally, Beauty Tech is implemented as an efficient platform in which the software maximizes the potential of the hardware.
The basic framework of Beauty Tech includes:
Capture an ImageThe process begins by capturing an image of the face—either with a dedicated Beauty Tech hardware device, or most commonly, a smartphone.
Process the ImageThe real power kicks in with the software used to process the image. This includes, among other factors, deciphering the elements of the face—establishing precisely where eyebrows begin and end, defining the eyes and surrounding skin, identifying the exact boundaries of the lips, and recognizing each strand of hair. Perfect Corp.’s AR beauty technology is based on the innovative real-time high-definition facial 3D live mesh with 3,900 meshes, ensuring the accuracy of the virtual try-on experience.
Apply AIThe best Beauty Tech solutions incorporate AI to bring greater precision into the facial mapping and tracking required to accomplish all of the above—and to enable these boundaries to be fluidly identified even as the user turns to see how a virtual try-on looks from different angles—providing a magic mirror experience.
Launch ARThe AR experience begins as mathematical algorithms are executed behind the scenes to change values—applying different colors to the lips, overlaying foundation, changing brow shapes, trying new hair colors, testing new eye shadow, or exploring various nail polish shades. Precision AR technology is required to achieve the real-world look and feel of an immersive AR experience.
BIG DataBig data—and more AI—can be used to further the personalization of Beauty Tech. Big Data and AI can be harnessed, for example, to power a foundation finder application that can better match a person’s skin than the user could. Of course, the user can try whatever foundation they like, but solid AI and Big Data can tap into thousands of skin types to generate optimal suggestions. Data can also be harvested from previous interactions, as an application learns the preferences of individual users—as well as the experimentation that similar users enjoy.
For example, Perfect Corp.’s brand partner MAKE UP FOR EVER offers AI-powered foundation shade matching on its e-commerce site, allowing consumers to find their ideal shade in a matter of seconds. This innovative solution offers a safe and secure way to experience the product, facilitating confident purchase decisions.
How To Build the Best Beauty Tech Experience
When evaluating the effectiveness of Beauty Tech platforms, some useful yardsticks include:
Facial TrackingA fully immersive AR experience depends upon faultless facial tracking. Users don’t sit still when working with a virtual try-on. They want to move their head about, just as they would in front of a mirror. The lack of faultless facial tracking can mean that lipstick lags behind when moving from side to side, or that a new hair color gets stuck on the wall when the user moves to see how a new hair color will look from a different angle. An enormous amount of technology goes into real-time facial tracking. Without this technology, makeup can take on a clownish appearance, and the session, while perhaps a bit amusing, won’t satisfy nor bring users back. Perfect Corp.’s patented AgileFace® tracking technology accurately detects all facial features and overlays virtual makeup on them with the utmost precision, even aligning virtual makeup with facial movement. This new technology properly identifies all variations of facial features, resulting in the most inclusive virtual try-on experience yet.
Precision virtual try-on for all categoriesThe value of a virtual try-on experience—for the customer as well as the seller—is to provide an authentic presentation of how a product will actually look on the customer. This can be especially challenging with hair color, as the color applied will differ according to the customer's natural hair color. “Lashes are extremely hard to capture in AR,” says American International Industries’ Marwan Zreik. “They can look pasted on. And the same is true for hair, which is also a major technical challenge to precisely render. We were all blown away by the accuracy of Perfect Corp’s AI and AR. The color went onto the hair in a completely natural way. It actually works with the tone of your natural hair color, which means that each of our Punky colors looks different on every person depending on their base hair color.”
Exacting Lip and Eyeliner ColorsAs with hair color, the value of a virtual try-on experience is completely dependent upon the ability to render in AR exactly how a product will look on the user. For lipstick, this means not only rendering colors precisely but also capturing the texture. If lipstick is glossy, or metallic, shimmers, or has other unique features, precision AR is required to enable the user to see exactly how it will look on their own lips. Failure at this results in unhappy customers and can have devastating effects on social media. Stila Cosmetics partnered with Perfect Corp. to offer precisely this kind of hyper-realistic virtual try-on experience on its website. Customers can instantly try on dozens of popular makeup products, including lipstick, eye shadow, mascara, highlighter, and more. The digital effects of a virtual makeup try-on so closely mimic those of a physical try-on that consumers can be confident they are making the right purchase decision.
High-Definition BrowsHair is generally considered one of the greatest challenges for AR because it needs to be rendered strand by strand. The challenge is even greater when working with brows. Without precise definition, the virtual try-on will be unsatisfactory. Again, facial tracking is important, too. If a user arches a brow, that brow must arch.
Foundation Is Often Puzzling For UsersJust which foundation is right? Finding the perfect foundation shade can be a challenging process for many consumers, but AI beauty tech can simplify the process. For example, Estée Lauder’s latest in-store beauty app combines precision foundation shade detection with a virtual try-on-color match experience. With iMatch™ Virtual Shade Expert, Perfect Corp.’s AI deep learning for real-time skin-tone detection uses its knowledge base of 89,969 skin tones to help customers find their preferred foundation shade in Estée Lauder’s number one foundation, Double Wear Stay-in-Place Makeup, and see it virtually applied in real-time. “We found a customer’s perfect foundation shade match is not always her preferred shade,” says Gen Obata, Vice President Global Retail Experience at Estée Lauder. “So iMatch™ Virtual Shade Expert answers this need. It finds her perfect shade and then provides options that are lighter, darker, cooler, or warmer so she can choose the shade she thinks suits her best.” This kind of personalization creates a strong customer bond.
Unique Virtual ManicureFinding that perfect nail polish shade can be a daunting task, considering how challenging it can be to try nail products at the store and especially online. Perfect Corp.’s AI-powered AR virtual try-on for nails greatly simplifies this undertaking by delivering a real-time ultra-realistic 3D live preview of the nail products directly on consumers’ hands, without messy clean ups and time-consuming application.
Innovative Approach to Men’s GroomingBeauty Tech can be beneficial for men as well. Latest AI & AR developments have made it easy for men to virtually experiment with different beard styles and colors, with no need to shave or commit to a new look. Perfect Corp.’s AI Beard technology allows users to virtually experience beard dye products and try different beard styles using any device with a camera before committing to a certain look. The advanced AI technology can also simulate beard removal, allowing men’s grooming customers to visualize how they would look clean-shaven.
Powerful AIAI has been mentioned throughout as it provides foundational support to great AR. In addition to its role in facial detection and tracking, and its powerful mining of Big Data, AI can power personalization that creates tight customer bonds. Jon Roman, Senior Vice President, Consumer Marketing and Online at Estée Lauder puts it this way: “The combination of precision shade matching with personalization not only increases conversions, but you are also delighting the customer because she’s really getting something that’s personalized for her. This kind of personalization really builds customer loyalty.” AI can also be used for ‘look transfer’ in which a user can capture the essence of a favorite celebrity or influencer, and map that look to their own face.
Delta E PrecisionWe have spoken repeatedly about the need to precisely match colors displayed in a virtual try-on experience with those found in real life. And we’ve spoken of the variability of colors when applied to someone’s unique hair or their own unique skin texture and color. While this may sound subjective, color variations can be measured down to the point of imperceptibility through the use of Delta E metrics. Delta E is a metric for calibrating how the human eye perceives color differences. The term delta comes from mathematics, meaning change in a variable or function. The suffix E references the German word Empfindung, which broadly means sensation. Generally, a Delta E value of 3 or less means that color differentiation can only be detected by careful study. A value of 2 or less is considered to be imperceptible to the human eye. With our Perfect Corp. technology, the Delta E value is often below 1.5—making for a seamless experience when moving from AR virtual try-on colors to real-world products, and helping every consumer find their perfect product match.
Multi-Product CapabilityFinally, a Beauty Tech platform should provide multi-product and multi-vendor capability. Don’t expect that consumers will download a separate app for each product they like. We all suffer from app overload, and it disrupts the AR experience to have to move from one company’s lip app to another company’s eye app to a hair color app. That’s where Perfect Corp.’s AR and AI technology comes in to provide a unified and fun shopping experience. With this technology, users can create complete looks and then make their purchases accordingly. Ekta Chopra, VP of Digital, at e.l.f., speaks of the joy of product discovery. “Looking at the full consumer journey, discovery is a huge element,” Chopra says. “You want to create a digital environment where they can discover new products, try them on, and make the purchase. Perfect Corp gives us the augmented reality technology to provide our consumers with an amazing, personalized, discovery experience.”
e.l.f. Cosmetics offers virtual try-on experience on its website and mobile app for an extensive range of makeup products, from eyeshadow palettes to lipsticks to highlighters. Since incorporating Perfect Corp.’s virtual makeup technology into its e-commerce journey, e.l.f. Cosmetics reported a 200% higher conversion rate – “from shopping to actually purchasing.”
Virtual try-on has proven to be so popular with users, and such a powerful sales tool, that it is only natural that AR has been embraced by the giants of social media, including YouTube, Snapchat, Instagram, and Pinterest.
Here’s a quick look at the different ways in which social media is integrating AR into their platforms:
- Google Search Virtual Try-On & YouTube Beauty Try-On. Google is incorporating AR try-on—in both Google Search and YouTube. For its well-known search engine, Google has partnered with Perfect Corp. and launched an AR try-on experience using YouCam Makeup’s AR virtual try-on technology, together with beauty brands like Estee Lauder, MAC Cosmetics, and others, allowing consumers to try on makeup SKUs (through Google Search) on themselves using the front-facing camera on mobile devices.
- YouTube is incorporating AR. Giving its 2 billion global viewers an entirely new way of experimenting with their favorite beauty products. Google/YouTube has entered into a strategic partnership with Perfect Corp. to integrate Perfect Corp.’s AR technology into YouTube’s InStream video ads—enabling brands to run augmented reality try-on over YouTube. This lets viewers, for example, virtually try on makeup in real-time as they watch their favorite MUA artists on YouTube’s platform. A great example is found with e.l.f. Cosmetics’ newest SRSLY Satin Lipstick Virtual Try-On InStream video ad on YouTube, featuring Internet-celebrity Jasmine Brown. As Jasmine tries out the different shades of e.l.f. lipsticks, the viewer can simply hit the “try on” button and experiment with the exact shades—or any others that catch their eye, from the YouCam Makeup app just below the video. Users can also shop for whatever they see, right from the app.
Natural beauty brand Jane Iredale also integrated Perfect Corp.’s AR virtual try-on technology into their YouTube True View ad units, creating a fun experience and allowing viewers to try-on makeup shades in real-time as they viewed. Ross Miller, VP of Digital at Jane Iredale Cosmetics noted the success of the ads, “The ad had a 51% view rate which was almost double what we expected to see. We were also excited to see that 11% of those who viewed the ad actually joined the fun by activating their own VTO experience to go along with the video. That’s a big number because viewer click-through for ads in general is usually less than 1%.”
- Snapchat’s AR Makeup — Bridging the Online and Offline Worlds. Snapchat, the pioneering social media app that lets everybody tell their story, has become one of the leading companies powering the AR revolution. Snapchat is a natural place for AR because of its large audience share for Generation Z and millennial users—who are known to be big spenders, representing a staggering $143 billion in purchasing power and representing the largest group of consumers in the United States. The power of AR is demonstrated with Snapchat’s successful collaboration with Perfect Corp. – considering its AR expertise, beauty brands can use Perfect Corp,’s virtual SKU platform (Perfect Console) to bring the AR experience to the Snapchat platform quickly and effortlessly.
According to Snap Inc., campaigns that include shoppable AR Lenses see 2.4x higher action intent lift (compared to 2020 Q3 average). Together, Snapchat and Perfect Corp. are helping beauty brands reach the next generation of beauty consumers with technology that reinvents the beauty shopping journey. This technology creates fun, entertaining experience and vehicle for beauty discovery on a platform where users are excited to interact with AR.
- Seamless Virtual Beauty Experience on Instagram. Meta’s Instagram is not trailing behind when it comes to interactive virtual beauty experiences. Considering the importance of social commerce for product discovery and experimentation, Instagram integrated AR shopping experiences powered by Perfect Corp. technology in different spaces, including Stories, Ads, Posts, and more. With AR shoppable posts, Instagram users can virtually try on and purchase beauty products directly from their feeds. This experience brings many significant benefits to beauty brands, including increased social commerce sales and better engagement with consumers.
New Beauty Tech Trends - A Look into the Future
As the beauty industry continues its’ digital transformation, beauty tech will evolve to play a bigger role in omnichannel shopping experiences. Here’s a glimpse into current and upcoming trends:
- Beauty Tech as a Key Driver of Customer Engagement. Beauty tech has become fundamental for a successful direct-to-consumer strategy. Advanced beauty technology is the key to helping brands understand, engage, and connect with their consumers better—especially during the rise of digital shopping experiences versus in-person.
- The Rise of Beauty GAN and Livestream Technologies. While AR and AI have become beauty tech staples, the next generation facial simulations and interactive technologies will be the advancements re-shaping the future consumer experience.
- The Generation of AI Skin Tech. Skin tech is on the rise as the most requested new service category. This spotlights the growing importance of AI innovation necessary to deliver personalized skin analysis and product recommendations.
- 360 Digital Solutions for the Omnichannel Shopping Journey. A successful direct-to-consumer strategy must digitalize product try-on, engage consumers with personalized experiences, and enable cohesive digital inter-activities across all consumer touchpoints including retail, web, in-app, and across social. Perfect Corp. is the only beauty tech provider that offers 360 digital solutions across the omni-channel shopping journey. Perfect Corp.’s advanced AR and AI technologies allow brands to provide superior shopping experiences and personalized product recommendations to each customer. Perfect Corp.’s technologies are also compatible and easily integrated into a wide variety of social platforms and eCommerce tools, making it the most seamless and dynamic solution to enhance the omnichannel shopping journey.
- Technology for Sustainability. The beauty industry is one of the biggest contributors of product waste. Beauty tech is at the core of sustainability in the beauty industry. By going virtual, beauty brands and retailers can lower their eco-footprint by minimizing product waste and production of unnecessary samples, as well as reduce overconsumption by offering tailored shopping experiences.
- Expansion into New Beauty Categories: As beauty technology continues to evolve, it will continue to expand to new spaces in the beauty industry. With expansion to nail color and men’s grooming, beauty tech is quickly becoming a must-have component of the beauty shopping journey across categories.
Digital transformation is hardly a new concept for beauty brands. Possessing enough knowledge about Beauty Tech and fully understanding its immense capabilities is the key factor to driving business success in this highly digitized world. As Beauty Tech is a fast-growing segment, it is crucial for beauty brands and retailers to keep up with the latest trends and developments. Furthermore, as readers have learned from this article, Beauty Tech is an indispensable resource for solving consumer pain points, creating a better beauty shopping experience, and ultimately accelerating business growth.
Get expert advice from Perfect Corp. and be part of the beauty tech trend. Contact us for details on beauty tech solutions trusted by 400+ brands globally.