Latent Optimisation

Latent Optimisation is a technique used for generative adversarial networks to refine the sample quality of $z$. Specifically, it exploits knowledge from the discriminator $D$ to refine the latent source $z$. Intuitively, the gradient $\nabla_{z}f\left(z\right) = \delta{f}\left(z\right)\delta{z}$ points in the direction that better satisfies the discriminator $D$, which implies better samples. Therefore, instead of using the randomly sampled $z \sim p\left(z\right)$, we uses the optimised latent:

$$ \Delta{z} = \alpha\frac{\delta{f}\left(z\right)}{\delta{z}} $$

$$ z' = z + \Delta{z} $$

Source: LOGAN .