Semantic Facial Expression Editing using Autoencoded Flow

Raymond Yeh¹   Ziwei Liu²   Dan B Goldman³   Aseem Agarwala³  

1. University of Illinois Urbana-Champaign   2. The Chinese University of Hong Kong   3. Google Inc.  

Abstract

High-level manipulation of facial expressions in images — such as changing a smile to a neutral expression — is challenging because facial expression changes are highly non-linear, and vary depending on the appearance of the face. We present a fully automatic approach to editing faces that combines the advantages of flow-based face manipulation with the more recent generative capabilities of Variational Autoencoders (VAEs). During training, our model learns to encode the flow from one expression to another over a low-dimensional latent space. At test time, expression editing can be done simply using latent vector arithmetic. We evaluate our methods on two applications: 1) single-image facial expression editing, and 2) facial expression interpolation between two images. We demonstrate that our method generates images of higher perceptual quality than previous VAE and flow-based methods.

Materials

[Paper]

Citation

@inproceedings{
    yeh2016semantic,
    title={Semantic Facial Expression Editing using Autoencoded Flow},
    author={Yeh, Raymond A. and Liu, Ziwei and Goldman, Dan B and Agarwala, Aseem},
    journal={arXiv preprint arXiv:1611.09961},
    year={2016}
}