Overall concept
eTryOn brings together the fashion industry (from fashion designers and consumers to fashion marketeers) with technology experts in various IT fields (3D modeling, garment fitting, AI, physics simulations, user profiling and recommendations, game engine development) in order to revolutionize the way people interact with fashion. In creating these new HFI solutions, the concept of eTryOn relies on technologies that can automate the content generation for fueling novel interaction devices (VR, AR, mobile phones).
More specifically, we research into technologies that can:
- Digitize full human bodies into 3D avatars using mainstream devices (i.e. smartphones),
- Automatically simulate and visualize how digitally designed garments fit and look on the avatars in terms of size, style and move,
- Extract fashion insights such as emerging trends using preference data from actual consumers and provide fashion recommendations to users.
By bringing together all these technologies in three innovative HFI applications, we can transform the creative, social and shopping experience that various types of users have when they interact with fashion items. In more detail, eTryOn’s conceptual layers and the challenges we aim to solve are the following:
Digital avatars: eTryOn will deliver a mobile application for generating personal 3D avatars that can be used for virtual try-ons. While the literature offers various solutions for 3D digitization of objects, developing avatars for fitting garments creates new requirements for exact body shapes, accurate measurements and attractive texturing. Furthermore, the interactive devices that typically require high-quality and high-resolution content for convincing photorealism increase the demands from the scanning technology in terms of avatar quality. Finally, generating such avatars and extracting exact body measurements using a portable device available to every consumer (i.e. mobile phone) is a significant challenge that eTryOn aims to take on.
Digital garments: Considering that lately more and more fashion designers have shifted to digitally designing garments in dedicated professional software, eTryOn will follow this trend and rely solely on garments with digital designs. In this direction, it is important that the developed solution is compatible with the current workflow of the designers so that the designed garments can be automatically transferred in a form compatible for digital tryons.
Avatar-garment simulations: A significant challenge in eTryOn is to produce realistic and stylistic tryons of garments on the avatars online and in real time. Existing designer software (e.g. CLO3D [CLO], Browzwear [BROWZ]) provide high quality artistic simulations, but they require both human intervention and significant time to process. On the other hand, gaming physics engines perform the simulation efficiently, but they do not incorporate stylistic elements and thus they do not provide the necessary information on how a garment fits. In eTryOn, our objective is to bring the best of these two approaches together by using the high-quality artistic simulations to drive the game engines so as to provide stylistics simulations automatically in close to real-time.
Fashion insights: The fashion industry has seen a major reform during the past years due to the widespread use of social media; from Instagram influencers and models to the creativity of the fashion designers. In eTryOn, our goal is to mine all the information coming from this new era of digital fashion in order to extract useful insights that can accompany our HFI applications. More specifically, we will rely on user preferences data from MLZ’s shopping application in order to extract the following insights: i) detection of emergent fashion trends to enhance and inform the creative process of garment design, ii) assessment of fashion product images with real users to provide insights on the expected impact of images with the digitally tried-on garments, iii) fashion item recommendations to consumers to assist in the online shopping experience.
HFI systems: Finally, after all the necessary components become available (3D avatars, garments, avatar-garment simulations, fashion insights), we combine them in novel systems that enhance the creative, social and shopping experience of the users. More specifically, in eTryOn, we deliver three new interaction paradigms: i) a virtual garment design application for the fashion designers, ii) a mobile application for the fashion consumer to use for their social media, and iii) an AR dressing room for virtually trying on clothes at home. These HFI systems will be demonstrated in the context of the three pilots described in the following section.