Image Processing Using Multi-Code GAN Prior
Jinjin Gu1  Yujun Shen2  Bolei Zhou2 
1The Chinese University of Hong Kong, Shenzhen
2The Chinese University of Hong Kong
[Paper]  [Code]
Overview
We propose multi-code GAN prior (mGANprior) to incorporate the well-trained GANs as effective prior to a variety of image processing tasks. In particular, we employ multiple latent codes to invert a fixed GAN model, and then introduce adaptive channel importance to compose the features maps from these codes at some intermediate layer of the generator. The resulting high-fidelity image reconstruction enables the trained GAN models as prior to many real-world applications, such as image colorization, super-resolution, image inpainting, and semantic manipulation.
Results
Various image processing tasks with mGANprior.
BibTeX
@inproceedings{gu2020image,
  title     = {Image Processing Using Multi-Code GAN Prior},
  author    = {Gu, Jinjin and Shen, Yujun and Zhou, Bolei},
  booktitle = {CVPR},
  year      = {2020}
}
Related Work
Y. Shen, J. Gu, X. Tang, B. Zhou. Interpreting the Latent Space of GANs for Semantic Face Editing. CVPR 2020.
Comment: Interprets the face semantics emerging in the latent space of GANs.
D. Bau, J. Y. Zhu, H. Strobelt, B. Zhou, J. B. Tenenbaum, W. T. Freeman, A. Torralba. GAN Dissection: Visualizing and Understanding Generative Adversarial Networks. ICLR 2019.
Comment: Interprets the intermediate representation of GANs.
D. Bau, J. Y. Zhu, J. Wulff, W. Peebles, H. Strobelt, B. Zhou, A. Torralba. Inverting layers of a large generator. ICLRW 2019.
Comment: Inverts GANs by layer-wise Encoder Networks.
D. Ulyanov, A. Vedaldi, V. Lempitsky. Deep Image Prior. CVPR 2018.
Comment: Uses deep neural networks (discriminative model) as structured image prior.
J. Y. Zhu, P. Krähenbühl, E. Shechtman, Alexei A. Efros. Generative Visual Manipulation on the Natural Image Manifold. ECCV 2016.
Comment: Manipulates images by learning the natural image manifold from GANs.