Generative Hierarchical Features from Synthesizing Images
Yinghao Xu*,  Yujun Shen*,  Jiapeng Zhu,  Ceyuan Yang,  Bolei Zhou
The Chinese University of Hong Kong
[Paper]  [Code]
Overview
In this work, we argue that the pre-trained GAN generator can be considered as a learned multi-scale loss. Training with it can bring highly competitive hierarchical and disentangled visual features, which we call Generative Hierarchical Features (GH-Feat). We further show that GH-Feat facilitates a wide range of not only generative but more importantly discriminative tasks, including face verification, landmark detection, layout prediction, transfer learning, style mixing, image editing, etc.
A Wide Range of Applications with GH-Feat

Discriminative Tasks

Indoor scene layout prediction

Facial landmark detection

Generative Tasks

Image harmonization

Local Editing

BibTeX
@inproceedings{xu2021generative,
  title     = {Generative Hierarchical Features from Synthesizing Images},
  author    = {Xu, Yinghao and Shen, Yujun and Zhu, Jiapeng and Yang, Ceyuan and Zhou, Bolei},
  booktitle = {CVPR},
  year      = {2021}
}
Related Work
T. Karras, S. Laine, T. Aila. A Style-Based Generator Architecture for Generative Adversarial Networks. CVPR 2019.
Comment: Proposes style-based generator for high-quality image synthesis.
J. Donahue and K. Simonyan. Large Scale Adversarial Representation Learning. NeurIPS 2019.
Comment: Employs BigGAN for representation learning by training an encoder together with the BigGAN generator.
S. Pidhorskyi, D. Adjeroh, G. Doretto. Adversarial Latent Autoencoders. CVPR 2020.
Comment: Learns an auto-encoder based on the StyleGAN structure for disentangled representation learning.
J. Zhu, Y. Shen, D. Zhao, B. Zhou. In-Domain GAN Inversion for Real Image Editing. ECCV 2020.
Comment: Applies well-trained StyleGAN models for generative tasks, such as real image editing.
K. He, H. Fan, Y. Wu, S. Xie, R. Girshick. Momentum Contrast for Unsupervised Visual Representation Learning. CVPR 2020.
Comment: Builds a dynamic dictionary with a queue and a moving-averaged encoder for contrastive learning.