Generative Models

## Beta-VAE

Introduced by Higgins et al. in beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework

Beta-VAE is a type of variational autoencoder that seeks to discovered disentangled latent factors. It modifies VAEs with an adjustable hyperparameter $\beta$ that balances latent channel capacity and independence constraints with reconstruction accuracy. The idea is to maximize the probability of generating the real data while keeping the distance between the real and estimated distributions small, under a threshold $\epsilon$. We can use the Kuhn-Tucker conditions to write this as a single equation:

$$\mathcal{F}\left(\theta, \phi, \beta; \mathbf{x}, \mathbf{z}\right) = \mathbb{E}_{q_{\phi}\left(\mathbf{z}|\mathbf{x}\right)}\left[\log{p}_{\theta}\left(\mathbf{x}\mid\mathbf{z}\right)\right] - \beta\left[D_{KL}\left(\log{q}_{\theta}\left(\mathbf{z}\mid\mathbf{x}\right)||p\left(\mathbf{z}\right)\right) - \epsilon\right]$$

where the KKT multiplier $\beta$ is the regularization coefficient that constrains the capacity of the latent channel $\mathbf{z}$ and puts implicit independence pressure on the learnt posterior due to the isotropic nature of the Gaussian prior $p\left(\mathbf{z}\right)$.

We write this again using the complementary slackness assumption to get the Beta-VAE formulation:

$$\mathcal{F}\left(\theta, \phi, \beta; \mathbf{x}, \mathbf{z}\right) \geq \mathcal{L}\left(\theta, \phi, \beta; \mathbf{x}, \mathbf{z}\right) = \mathbb{E}_{q_{\phi}\left(\mathbf{z}|\mathbf{x}\right)}\left[\log{p}_{\theta}\left(\mathbf{x}\mid\mathbf{z}\right)\right] - \beta{D}_{KL}\left(\log{q}_{\theta}\left(\mathbf{z}\mid\mathbf{x}\right)||p\left(\mathbf{z}\right)\right)$$

#### Latest Papers

PAPER DATE
Full Encoder: Make Autoencoders Learn Like PCA
Zhouzheng LiKun Feng
2021-03-25
AI Discovering a Coordinate System of Chemical Elements: Dual Representation by Variational Autoencoders
Alex Glushkovsky
2020-11-24
Unsupervised anomaly localization using VAE and beta-VAE
Leixin ZhouWenxiang DengXiaodong Wu
2020-05-19
AI Giving Back to Statistics? Discovery of the Coordinate System of Univariate Distributions by Beta Variational Autoencoder
Alex Glushkovsky
2020-04-06
Variational Learning with Disentanglement-PyTorch
| Amir H. AbdiPurang AbolmaesumiSidney Fels
2019-12-11
Information bottleneck through variational glasses
Slava VoloshynovskiyMouad KondahShideh RezaeifarOlga TaranTaras HolotyakDanilo Jimenez Rezende
2019-12-02
Flatsomatic: A Method for Compression of Somatic Mutation Profiles in Cancer
Geoffroy Dubourg-FelonneauYasmeen KussadDominic KirkhamJohn W CassidyNirmesh PatelHarry W Clifford
2019-11-27
A Preliminary Study of Disentanglement With Insights on the Inadequacy of Metrics
Amir H. AbdiPurang AbolmaesumiSidney Fels
2019-11-26
Disentangling Interpretable Generative Parameters of Random and Real-World Graphs
| Niklas StoehrEmine YilmazMarc BrockschmidtJan Stuehmer
2019-10-12
ISA-VAE: Independent Subspace Analysis with Variational Autoencoders
Jan StühmerRichard TurnerSebastian Nowozin
2019-05-01
IB-GAN: Disentangled Representation Learning with Information Bottleneck GAN
Insu JeonWonkwang LeeGunhee Kim
2019-05-01
The Information-Autoencoding Family: A Lagrangian Perspective on Latent Variable Generative Modeling
Shengjia ZhaoJiaming SongStefano Ermon
2018-01-01
beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework
| Irina HigginsLoic MattheyArka PalChristopher BurgessXavier GlorotMatthew BotvinickShakir MohamedAlexander Lerchner
2017-04-26

#### Tasks

TASK PAPERS SHARE
Anomaly Detection 1 25.00%
Graph Embedding 1 25.00%
Graph Generation 1 25.00%
Graph Representation Learning 1 25.00%

#### Components

COMPONENT TYPE
🤖 No Components Found You can add them if they exist; e.g. Mask R-CNN uses RoIAlign