A Simple Unified Information Regularization Framework for Multi-Source Domain Adaptation

Adversarial learning strategy has demonstrated remarkable performance in dealing with single-source unsupervised Domain Adaptation (DA) problems, and it has recently been applied to multi-source DA problems. A potential pitfall of the existing DA methods that use multiple domain discriminators for each source domain is that domain-discriminative information is inevitably distributed across multiple discriminators. Despite this issue, the effect of using multiple discriminators on the quality of latent space representations has been poorly understood. To fully address this issue, we situate adversarial DA in the context of information regularization. First, we present a unified information regularization framework for multi-source DA. Our framework shows that the information shared across domains cannot be gleaned with multiple discriminators. It further provides a theoretical justification for using a single and unified domain discriminator to encourage the synergistic integration of the information gleaned from each domain. Second, this motivates us to implement a novel neural architecture called a Multi-source Information-regularized Adaptation Networks (MIAN). The proposed model significantly reduces the variance of stochastic gradients and increases computational-efficiency. Large-scale simulations on various multi-source DA scenarios demonstrate that MIAN, despite its structural simplicity, reliably outperforms other state-of-the-art methods by a large margin. Our work offers the possibility of garnering fundamental insights from multiple domains into the development of highly generalizable algorithms.