Please use this identifier to cite or link to this item:
Title: CNN-based real-time dense face reconstruction with inverse-rendered photo-realistic face images
Authors: Guo, Yudong
Zhang, Juyong
Cai, Jianfei
Jiang, Boyi
Zheng, Jianmin
Keywords: Engineering::Computer science and engineering
Issue Date: 2019
Source: Guo, Y., Zhang, J., Cai, J., Jiang, B., & Zheng, J. (2019). CNN-based real-time dense face reconstruction with inverse-rendered photo-realistic face images. IEEE transactions on pattern analysis and machine intelligence, 41(6), 1294-1307. doi:10.1109/TPAMI.2018.2837742
Journal: IEEE transactions on pattern analysis and machine intelligence
Abstract: With the powerfulness of convolution neural networks (CNN), CNN based face reconstruction has recently shown promising performance in reconstructing detailed face shape from 2D face images. The success of CNN-based methods relies on a large number of labeled data. The state-of-the-art synthesizes such data using a coarse morphable face model, which however has difficulty to generate detailed photo-realistic images of faces (with wrinkles). This paper presents a novel face data generation method. Specifically, we render a large number of photo-realistic face images with different attributes based on inverse rendering. Furthermore, we construct a fine-detailed face image dataset by transferring different scales of details from one image to another. We also construct a large number of video-type adjacent frame pairs by simulating the distribution of real video data.11.All these coarse-scale and fine-scale photo-realistic face image datasets can be downloaded from With these nicely constructed datasets, we propose a coarse-to-fine learning framework consisting of three convolutional networks. The networks are trained for real-time detailed 3D face reconstruction from monocular video as well as from a single image. Extensive experimental results demonstrate that our framework can produce high-quality reconstruction but with much less computation time compared to the state-of-the-art. Moreover, our method is robust to pose, expression and lighting due to the diversity of data.
ISSN: 0162-8828
DOI: 10.1109/TPAMI.2018.2837742
Rights: © 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. The published version is available at: 10.1109/TPAMI.2018.2837742
Fulltext Permission: open
Fulltext Availability: With Fulltext
Appears in Collections:SCSE Journal Articles


Updated on Jan 23, 2021


Updated on Jan 24, 2021

Page view(s)

Updated on Jan 27, 2021

Google ScholarTM




Items in DR-NTU are protected by copyright, with all rights reserved, unless otherwise indicated.