Hyeongwoo Kim1,2    Michael Zollhöfer1,2,3    Ayush Tewari1,2    Justus Thies4    Christian Richardt5    Christian Theobalt1,2

1 MPI Informatik       2 Saarland Informatics Campus       3 Stanford University       4 TU Munich       5 University of Bath

Computer Vision and Pattern Recognition (CVPR) 2018


Abstract

We introduce InverseFaceNet, a deep convolutional inverse rendering framework for faces that jointly estimates facial pose, shape, expression, reflectance and illumination from a single input image. By estimating all parameters from just a single image, advanced editing possibilities on a single face image, such as appearance editing and relighting, become feasible in real time. Most previous learning-based face reconstruction approaches do not jointly recover all dimensions, or are severely limited in terms of visual quality. In contrast, we propose to recover high-quality facial pose, shape, expression, reflectance and illumination using a deep neural network that is trained using a large, synthetically created training corpus. Our approach builds on a novel loss function that measures model-space similarity directly in parameter space and significantly improves reconstruction accuracy. We further propose a self-supervised bootstrapping process in the network training loop, which iteratively updates the synthetic training corpus to better reflect the distribution of real-world imagery. We demonstrate that this strategy outperforms completely synthetically trained networks. Finally, we show high-quality reconstructions and compare our approach to several state-of-the-art approaches.

Downloads

Copyright

© Copyrights by the Authors, 2018. This is the authors’ version of the work. It is posted here for your personal use. Not for redistribution. The definitive version will be published in IEEE Xplore.

Bibtex

@inproceedings{InverseFaceNet,
  author    = {Hyeongwoo Kim and Michael Zollh{\"o}fer and Ayush Tewari and Justus Thies and Christian Richardt and Christian Theobalt},
  title     = {{InverseFaceNet}: Deep Monocular Inverse Face Rendering},
  booktitle = {Conference on Computer Vision and Pattern Recognition (CVPR)},
  year      = {2018},
  month     = jun,
  url       = {http://richardt.name/publications/inversefacenet/},
}