In-Domain GAN Inversion for Real Image Editing
Jiapeng Zhu1*,  Yujun Shen1*,  Deli Zhao2Bolei Zhou1
1The Chinese University of Hong Kong         2Xiaomi AI Lab
Overview
In this work, we argue that the GAN inversion task is required not only to reconstruct the target image by pixel values, but also to keep the inverted code in the semantic domain of the original latent space of well-trained GANs. For this purpose, we propose In-Domain GAN inversion (IDInvert) by first training a novel domain-guided encoder which is able to produce in-domain latent code, and then performing domain-regularized optimization which involves the encoder as a regularizer to land the code inside the latent space when being finetuned. The in-domain codes produced by IDInvert enable high-quality real image editing with fixed GAN models.
Results
Semantic diffusion results.
Image editing results.
Check more results in the following video.
BibTeX
@inproceedings{zhu2020indomain,
  title     = {In-domain GAN Inversion for Real Image Editing},
  author    = {Zhu, Jiapeng and Shen, Yujun and Zhao, Deli and Zhou, Bolei},
  booktitle = {Proceedings of European Conference on Computer Vision (ECCV)},
  year      = {2020}
}
Related Work
Y. Shen, J. Gu, X. Tang, B. Zhou. Interpreting Latent Space of GANs for Semantic Face Editing. CVPR 2020.
Comment: Proposes a technique for semantic face editing in latent space.
J.Y. Zhu, P. Krähenbühl, E. Shechtman, A. A. Efros. Generative Visual Manipulation on the Natural Image Manifold. ECCV 2016.
Comment: Proposes a method for realistic photo manipulation and a system for interactive drawing using GANs.
J. Gu, Y. Shen, B. Zhou. Image Processing Using Multi-Code GAN Prior. CVPR 2020.
Comment: Employs multiple latent codes to invert a GAN model as prior for real image processing.