Digital makeup using machine learning algorithms

Abstract

In this report, we present a pipeline system of digital makeup for industry scenarios. The pipeline contains two parts: i) facial feature semantic segmentation; ii) colour transfer. For facial feature semantic segmentation task, we adopt fully convolutional network (FCN) with weighted cross entropy as loss function during training; for colour transfer task, we experimented N-dimensional Probability Density Function transfer algorithm, a fast exemplar-based image colourisation approach using colour embeddings named Color2Embed, and deep exemplar-bases colourisation approach. Considering economical and qualitative factors, we conclude that model trained by VGG16 FCN with weighted cross entropy together with fast exemplar-based image colourisation yields the most suitable result.