In-Domain GAN Inversion for Real Image Editing
Jiapeng Zhu1*,  Yujun Shen1*,  Deli Zhao2,  Bolei Zhou1
1The Chinese University of Hong Kong         2Xiaomi AI Lab
[Paper]  [Code (TensorFlow)]  [Code (PyTorch)] [Colab]
Overview
In this work, we argue that the GAN inversion task is required not only to reconstruct the target image by pixel values, but also to keep the inverted code in the semantic domain of the original latent space of well-trained GANs. For this purpose, we propose In-Domain GAN inversion (IDInvert) by first training a novel domain-guided encoder which is able to produce in-domain latent code, and then performing domain-regularized optimization which involves the encoder as a regularizer to land the code inside the latent space when being finetuned. The in-domain codes produced by IDInvert enable high-quality real image editing with fixed GAN models.
Results
Semantic diffusion results.
Image editing results.
See more results in the following demo video:

This work is featured in Two Minute Papers Youtube channel as below:
BibTeX
@inproceedings{zhu2020indomain,
  title     = {In-domain GAN Inversion for Real Image Editing},
  author    = {Zhu, Jiapeng and Shen, Yujun and Zhao, Deli and Zhou, Bolei},
  booktitle = {Proceedings of European Conference on Computer Vision (ECCV)},
  year      = {2020}
}
Related Work
Y. Shen, J. Gu, X. Tang, B. Zhou. Interpreting Latent Space of GANs for Semantic Face Editing. CVPR 2020.
Comment: Proposes a technique for semantic face editing in latent space.
C. Yang, Y. Shen, B. Zhou. Semantic Hierarchy Emerges in Deep Generative Representations for Scene Synthesis. arXiv preprint arXiv:1911.09267, 2019
Comment: Explores the emergent semantic hierarchy in scene synthesis models.
R. Abdal, Y. Qin, P. Wonka Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space? ICCV 2019.
Comment: Explores how to Embed Images into the latent space.
P. Isola, J.Y. Zhu, T. Zhou, A. A. Efros. Image-to-Image Translation with Conditional Adversarial Nets. CVPR 2017.
Comment: Investigates image-to-image translation using conditional GANs.
J.Y. Zhu, P. Krähenbühl, E. Shechtman, A. A. Efros. Generative Visual Manipulation on the Natural Image Manifold. ECCV 2016.
Comment: Proposes a method for realistic photo manipulation and a system for interactive drawing using GANs.
J. Gu, Y. Shen, B. Zhou. Image Processing Using Multi-Code GAN Prior. CVPR 2020.
Comment: Employs multiple latent codes to invert a GAN model as prior for real image processing.