Closed-Form Factorization of
Latent Semantics in GANs
Yujun ShenBolei Zhou
The Chinese University of Hong Kong
Overview
In this work, we propose a closed-form algorithm, called SeFa, for unsupervised latent Semantics Factorization in GANs. More concretely, we investigate the very first fully-connected layer used in the GAN generator. We argue that this layer actually filters out some negligible directions in the latent space and highlights the directions that are critical for image synthesis. By finding these important directions, we are able to identify versatile semantics across various types of GAN models with an extremely fast implementation (i.e., less than 1 second).

Fun Animations
The following animations are created by manipulating the versatile semantics unsupervisedly found by SeFa from GAN models trained on various datasets.

Anime Faces

Pose Mouth Eye

Cats

Posture (Left & Right) Posture (Up & Down) Zoom

Cars

Orientation Vertical Position Shape

Below shows the full demo video of our manipulation interface using SeFa.
BibTeX
@article{shen2020closedform,
  title   = {Closed-Form Factorization of Latent Semantics in GANs},
  author  = {Shen, Yujun and Zhou, Bolei},
  journal = {arXiv preprint arXiv:2007.06600},
  year    = {2020}
}
Related Work
Y. Shen, J. Gu, X. Tang, B. Zhou. Interpreting the Latent Space of GANs for Semantic Face Editing. CVPR 2020.
Comment: Interprets the face semantics emerging in the latent space of GANs with the help of off-the-shelf classifiers.
L. Goetschalckx, A. Andonian, A. Oliva, P. Isola. GANalyze: Toward Visual Definitions of Cognitive Image Properties. ICCV 2019.
Comment: Controls the latent space of GANs to increase the memorability of synthesized images.
A. Jahanian, L. Chai, P. Isola. On the "Steerability" of Generative Adversarial Networks. ICLR 2020.
Comment: Shifts the data distribution by steering the latent code to fit camera movements and color changes.
A. Voynov and A. Babenko. Unsupervised Discovery of Interpretable Directions in the GAN Latent Space. ICML 2020.
Comment: Interprets meaningful directions in GAN latent space by unsupervisedly training a direction reconstructor.