Yi Zhang General Exam Presentation TODAY Tuesday, May 29, 2018 10:00 am CS 302 Committee: Prof. Sanjeev Arora (advisor), Prof. Elad Hazan, Prof. Sebastian Seung Title: Theory and Practice of Generative Adversarial Networks (GANs) Abstract: I will discuss the (in)ability of GANs to learn the data generating distribution. A theoretical analysis of various GAN architectures will be presented, along with a principled and practical test for the support size of Generator’s distribution. In the first part of my talk, I'll show training of GAN architectures may appear successful but the trained distribution may be far from target distribution in standard metrics. The presented theory prophecies the notorious “mode collapse” failure mode of GANs. It is also shown that an approximate pure equilibrium exists in the discriminator/generator game for a special class of generators with natural training objectives when generator capacity and training set sizes are moderate. This existence of equilibrium inspires MIX+GAN protocol, which can be combined with any existing GAN training, and empirically shown to improve some of them. In the second part, I’ll extend the theoretical analysis to more recent encoder-decoder GANs architectures (e.g., BiGAN/ALI), which were proposed to learn more meaningful features via GANs and (consequently) to also solve the mode-collapse issue. Specifically, I’ll show that such encoder-decoder training objectives still cannot guarantee learning of the full distribution. More seriously, they cannot prevent learning meaningless codes for data, contrary to usual intuition. In addition, I’ll also propose a novel test for estimating support size using the birthday paradox of discrete probability. Using this, empirical evidence is presented that well-known GANs approaches do learn distributions of fairly low support. Barbara A. Mooring Interim Graduate Coordinator Computer Science Department Princeton University