Generative Hierarchical Features from Synthesizing Images
Yinghao Xu*,  Yujun Shen*,  Jiapeng ZhuCeyuan YangBolei Zhou
The Chinese University of Hong Kong
In this work, we argue that the pre-trained GAN generator can be considered as a learned multi-scale loss. Training with it can bring highly competitive hierarchical and disentangled visual features, which we call Generative Hierarchical Features (GH-Feat). We further show that GH-Feat facilitates a wide range of not only generative but more importantly discriminative tasks, including face verification, landmark detection, layout prediction, transfer learning, style mixing, image editing, etc.
A Wide Range of Applications with GH-Feat

Discriminative Tasks

Indoor scene layout prediction

Facial landmark detection

Generative Tasks

Image harmonization

Local Editing

  title     = {Generative Hierarchical Features from Synthesizing Images},
  author    = {Xu, Yinghao and Shen, Yujun and Zhu, Jiapeng and Yang, Ceyuan and Zhou, Bolei},
  booktitle = {CVPR},
  year      = {2021}
Related Work
J. Donahue and K. Simonyan. Large Scale Adversarial Representation Learning. NeurIPS 2019.
Comment: Employs BigGAN for representation learning by training an encoder together with the BigGAN generator.
S. Pidhorskyi, D. Adjeroh, G. Doretto. Adversarial Latent Autoencoders. CVPR 2020.
Comment: Learns an auto-encoder based on the StyleGAN structure for disentangled representation learning.
J. Zhu, Y. Shen, D. Zhao, B. Zhou. In-Domain GAN Inversion for Real Image Editing. ECCV 2020.
Comment: Applies well-trained StyleGAN models for generative tasks, such as real image editing.
K. He, H. Fan, Y. Wu, S. Xie, R. Girshick. Momentum Contrast for Unsupervised Visual Representation Learning. CVPR 2020.
Comment: Builds a dynamic dictionary with a queue and a moving-averaged encoder for contrastive learning.