Information Bottleneck: Exact Analysis of (Quantized) Neural Networks

1 code implementation • ICLR 2022 • Stephan Sloth Lorenzen, Christian Igel, Mads Nielsen

In this setting, we observed a fitting phase for all layers and a compression phase for the output layer in all experiments; the compression in the hidden layers was dependent on the type of activation function.

On PAC-Bayesian Bounds for Random Forests

1 code implementation • 23 Oct 2018 • Stephan Sloth Lorenzen, Christian Igel, Yevgeny Seldin

This effect provides a significant boost in performance when the errors are independent or negatively correlated, but when the correlations are strong the advantage from taking the majority vote is small.

Generalization Bounds