On the Optimality of Generative Adversarial Networks — A Variational Perspective

Abstract

Generative adversarial networks (GANs) are a popular learning framework to model the underlying distribution of images. GANs comprise a min-max game between the generator and the discriminator. While the generator transforms noise into realistic images, the discriminator learns to distinguish between the reals and the fakes. GANs are trained to either minimize a divergence function or an integral probability metrics (IPMs). In this thesis, we focus on understanding the optimality of GAN discriminator, generator, and its inputs, viewed from the perspective of Variational Calculus. Considering both divergence- and IPM-minimizing GANs, with and without gradient-based regularizers, we analyze the optimality of the GAN discriminator. We show that the optimal discriminator solves the Poisson partial differential equation, and derive solutions involving Fourier-series and radial basis function expansions. We show that providing the generator with data coming from a closely related input datasets accelerates and stabilizes training even in scenarios where there is no visual similarity between the source and target datasets. To identify closely related datasets, we propose the “signed Inception distance” (SID) as a novel GAN measure. Through the variational formulation, we demonstrate that the the optimal generator in GANs is linked to score-based Langevin diffusion and gradient flows. Leveraging these insights, we explore training GANs with flow-based and score-based costs, and diffusion models that perform discriminator-based updates.