no code implementations • ICLR 2019 • Adar Elad, Doron Haviv, Yochai Blau, Tomer Michaeli
The recently proposed information bottleneck (IB) theory of deep nets suggests that during training, each layer attempts to maximize its mutual information (MI) with the target labels (so as to allow good prediction accuracy), while minimizing its MI with the input (leading to effective compression and thus good generalization).
