Mathematics of GANs (generative adversarial networks)

Question

Generative Adversarial Networks were introduced in http://papers.nips.cc/paper/5423-generative-adversarial-nets and has more than 20000 citations.

The paper introduced key paradigm changes which require applications from modern areas of mathematics. I wanted to ask what are some the mathematics required to understand GANs as far as we know now and what are some key resources which provide accessible insight and a roadmap to learning?

Answer 1

• Concerning the question asked in the comment: what is the "key paradigm" of a GAN:
The basic problem that a GAN seeks to solve is to find the probability distribution $\mu$ given a finite number of samples, via iterative improvement of a trial distribution $\nu$. So we need a way to represent a probability distribution (a generator) and a way to measure differences between two distribution functions (a discriminator). The "key paradigm" of a GAN is to model both the generator and the discriminator by a neural network.

• Concerning the question in the OP on the mathematics required to understand a GAN:
A Mathematical Introduction to Generative Adversarial Nets (GAN) (2020) is a recent overview.

Answer 2

I recommend some papers by Lars Mescheder:

• The paper The Numerics of GANs formalizes GANs as two-player games and analyzes their training dynamics,
• the paper Which Training Methods for GANs do actually Converge? takes this analysis further and
• the PhD thesis Stability and Expressiveness of Deep Generative Models cantains a quite thorough and mathematical introduction to GANs.