Collaborative Normalization for Unsupervised Domain Adaptation

1 Jan 2021  ·  Haifeng Xia, Taotao Jing, Zhengming Ding ·

Batch Normalization (BN) as an important component assists Deep Neural Networks achieving promising performance for extensive learning tasks by scaling distribution of feature representations within mini-batches. However, the application of BN suffers from performance degradation under the scenario of Unsupervised Domain Adaptation (UDA), since the estimated statistics fail to concurrently describe two different domains. In this paper, we develop a novel normalization technique, named Collaborative Normalization (CoN), for eliminating domain discrepancy and accelerating the model training of neural networks for UDA. Unlike typical strategies only exploiting domain-specific statistics during normalization, our CoN excavates cross-domain knowledge and simultaneously scales features from various domains by mimicking the merits of collaborative representation. Our CoN can be easily plugged into popular neural network backbones for cross-domain learning. One the one hand, \textbf{theoretical analysis} guarantees that models with CoN promote discriminability of feature representations and accelerate convergence rate; on the other hand, \textbf{empirical study} verifies that replacing BN with CoN in popular network backbones effectively improves classification accuracy with \textbf{4\%} in most learning tasks across three cross-domain visual benchmarks.

PDF Abstract

Results from the Paper


  Submit results from this paper to get state-of-the-art GitHub badges and help the community compare results to other papers.

Methods


No methods listed for this paper. Add relevant methods here