Projection Discriminator

A Projection Discriminator is a type of discriminator for generative adversarial networks. It is motivated by a probabilistic model in which the distribution of the conditional variable $\textbf{y}$ given $\textbf{x}$ is discrete or uni-modal continuous distributions.

If we look at the original solution for the loss function $\mathcal{L}_{D}$ in the vanilla GANs, we can decompose it into the sum of two log-likelihood ratios:

$$ f^{*}\left(\mathbf{x}, \mathbf{y}\right) = \log\frac{q\left(\mathbf{x}\mid{\mathbf{y}}\right)q\left(\mathbf{y}\right)}{p\left(\mathbf{x}\mid{\mathbf{y}}\right)p\left(\mathbf{y}\right)} = \log\frac{q\left(\mathbf{y}\mid{\mathbf{x}}\right)}{p\left(\mathbf{y}\mid{\mathbf{x}}\right)} + \log\frac{q\left(\mathbf{x}\right)}{p\left(\mathbf{x}\right)} = r\left(\mathbf{y\mid{x}}\right) + r\left(\mathbf{x}\right) $$

We can model the log likelihood ratio $r\left(\mathbf{y\mid{x}}\right)$ and $r\left(\mathbf{x}\right)$ by some parametric functions $f_{1}$ and $f_{2}$ respectively. If we make a standing assumption that $p\left(y\mid{x}\right)$ and $q\left(y\mid{x}\right)$ are simple distributions like those that are Gaussian or discrete log linear on the feature space, then the parametrization of the following form becomes natural:

$$ f\left(\mathbf{x}, \mathbf{y}; \theta\right) = f_{1}\left(\mathbf{x}, \mathbf{y}; \theta\right) + f_{2}\left(\mathbf{x}; \theta\right) = \mathbf{y}^{T}V\phi\left(\mathbf{x}; \theta_{\phi}\right) + \psi\left(\phi(\mathbf{x}; \theta_{\phi}); \theta_{\psi}\right) $$

where $V$ is the embedding matrix of $y$, $\phi\left(·, \theta_{\phi}\right)$ is a vector output function of $x$, and $\psi\left(·, \theta_{\psi}\right)$ is a scalar function of the same $\phi\left(\mathbf{x}; \theta_{\phi}\right)$ that appears in $f_{1}$. The learned parameters $\theta = ${$V, \theta_{\phi}, \theta_{\psi}$} are trained to optimize the adversarial loss. This model of the discriminator is the projection.