Deep Dirichlet Process Mixture Models

In this paper we propose the deep Dirichlet process mixture (DDPM) model, which is an unsupervised method that simultaneously performs clustering and feature learning. As a member of the Bayesian nonparametrics family, the traditional Dirichlet process mixture model is able to adapt the number of its mixture components. However its modelling capacity is restricted since the clustering is performed in the raw feature space, rendering it inapplicable to complex domains like images and texts. Our method alleviates this limitation by using the flow-based generative model, which is a deep invertible neural network, to learn more expressive features. These two seemly orthogonal models are unified by the Monte Carlo expectation-maximization algorithm, and during its iterations Gibbs sampling is used to generate samples from the posterior. This combination allows our method to exploit the mutually beneficial relation between clustering and feature learning. We conducted comparison experiments on four clustering benchmark datasets. The clustering performance of DDPM shows a significant gain over DPM in most cases and is competitive compared to other popular methods. Furthermore, the learned representation of DDPM is shown to be efficient and universal to boost other methods' performance.