Researchers from the University of California San Diego and Adobe Research have demonstrated how artificial intelligence and neural networking could one day go beyond making recommendations about what fashion to buy, to helping retailers and apparel makers design bespoke clothing for consumers based on what they can learn from a buyer's preferences.
Julian McAuley
Their findings were published in early November in a paper* on "Visually-Aware Fashion Recommendation and Design with Generative Image Models." Computer Science and Engineering (CSE) professor Julian McAuley and his second-year Ph.D. student, Wang-Cheng Kang, teamed on the research with industry experts Chen Fang and Zhaowen Wang from Adobe Research.
"This suggests a new type of recommendation approach that can be used for recommendation, production and design," said CSE's McAuley and his colleagues. "These frameworks can lead to richer forms of recommendation, where content recommendation and content generation are more closely linked."
The project aimed to test how well tools from artificial intelligence (AI) and machine learning can help the fashion industry and consumers -- particularly those among the growing cohort of shoppers eager to buy clothing on the Internet.
achieved by using a 'visually aware' recommender based
on Siamese CNNs; generation is achieved by using a
Generative Adversarial Net (GAN) to synthesize new
clothing items in the user's personal style. (Icons made by
Madebyoliver, Roundicons and Freepik from
www.flaticon.com)
While there are many algorithms and tools to help online retailers recommend designs to potential buyers, the UC San Diego-Adobe Research team went a giant step further. They wanted to see if it would be possible to crunch preference and other data not only to make recommendations, but potentially to enable computers to produce new clothing designs that would have an edge because they reflect a consumer's individual preferences.
Initially, the researchers focused on devising a system to create better recommendations, particularly in the case of 'visual' recommendations (where consumers can be swayed by how the product looks (as in the case of fashion apparel or artworks).
"Building effective recommender systems for domains like fashion is challenging due to the high level of subjectivity and the semantic complexity of the features involved," according to the new paper. They went on to note that visual recommendationcan be made more accurate by incorporating visual signals directly into the recommendation objective, using off-the-shelf feature representations derived from deep networks.
The team demonstrated that recommendation performance can be significantly improved by learning 'fashion aware' image representations directly, by training the image representation (from the pixel level) and the recommender system jointly. The paper grew out of recent work using Siamese Convolutional Neural Networks (Siamese-CNNs), so-called because they are a class of neural network architectures containing two or more identical subnetworks. (Siamese-CNNs are popularly used to find similarity or relationships between two comparable items.)
process for a given user and product category.
At left are real images; at right, synthetic ones.
The values are preference scores for each image.
For the fashion project, the researchers trained the Siamese-CNN to learn and classify a user's preferences for certain items.
From there, they used a neural networking framework called Generative Adversarial Network (GAN) to learn the distribution of fashion images and generate novel fashion items that maximize users' preferences. GANs train two networks on one set of data, and they have been particularly well-suited to generate realistic images. The resulting system can suggest items to buy from existing designs, but it can also be used to modify existing items, or to generate new designs tailored to a specific individual's preferences (based on 'big data ' about prior purchases, surveys, etc.).
"We show that our model can be used generatively, i.e., given a user and a product category, we can generate new images (in this case clothing items) that are most consistent with the user's personal taste," said first-author and CSE Ph.D. student Wang-Cheng Kang. "This represents a first step toward building systems that go beyond recommending existing items from a product corpus, to suggesting styles and helping to design new products.
The use of AI in the fashion industry is still in its infancy, but two of the world's largest online retailers -- Amazon and China's Alibaba -- are already working with AI tools, including GANs. As for the UC San Diego-Adobe Research project, the quality of the algorithmically-designed new clothes remains rudimentary at best. As CSE's McAuley told a writer from MIT Technology Review, "you'd have to read the tea leaves a little bit if you want to call that style or not." But, he warned, bringing neural networking to the fashion world is in its infancy.
______________________________
*Kang, W. C., Fang, C., Wang, Z., & McAuley, J. (2017). "Visually-Aware Fashion Recommendation and Design with Generative Image Models", arXiv preprint, arXiv:1711.02231.