Publications

In this paper, we present an interactive Virtual Reality (VR) application for fashion designers that aims to offer an immersive environment for accurately visualizing and testing garments in the design process. The VR environment simulates a designer’s fitting room, offering close up inspection of an avatar wearing a garment in a plethora of movements. This, in turn, offers to the designer a better perspective of how the final product will look, fit and behave. In implementing the proposed application, we first review the option in game engines and libraries to support a realistic avatar garment simulation, and propose a methodology for transforming the production-designed garments into a format that can be used in game engine cloth simulations.

In this paper, we present Snapwear, a Snapchat AR filter for visualizing real-world clothing in a magic mirror-alike experience with the use of a smartphone. Our objective is to document the steps needed to transform a garment designed for production to be ready-for-use in an AR environment. In this direction, we analyse the generation pipeline of an average avatar, the garment export from the fashion design software, its skinning and rigging to the average avatar and the simulation options offered by Snapchat. Finally, we discuss the AR experience creation process ranging from technical challenges such us body tracking and dealing with occlusions, to enabling the user interaction with the filter.

Fine-grained information extraction from fashion imagery is a challenging task due to the inherent diversity and complexity of fashion categories and attributes. Additionally, fashion imagery often depict multiple items while fashion items tend to follow hierarchical relations among various object types, categories and attributes. In this study, we address both issues with a 2-step hierarchical deep learning pipeline consisting of (1) a low granularity object type detection module (upper-body, lower-body, full-body, footwear) and (2) two classification modules for garment categories and attributes based on the outcome of the first step. For the category and attribute-level classification stages we examine a hierarchical label sharing (HLS) technique in two settings: (1) single-task learning (STL w/ HLS) and (2) multi-task learning with RNN and visual attention (MTL w/ RNN+VA). Our approach enables progressively focusing on appropriately detailed features for automatically learning the hierarchical relations of fashion and enabling predictions on images with complete outfits.
Empirically, STL w/ HLS reached 93.99% top-3 accuracy while MTL w/ RNN+VA reached 97.57% top-5 accuracy for category classification on the DeepFashion benchmark, surpassing the current state-of-the-art without requiring landmark or mask annotations nor specialised domain expertise.