Separation of Scales and a Thermodynamic Description of Feature Learning in Some CNNs

no code implementations • 31 Dec 2021 • Inbar Seroussi, Gadi Naveh, Zohar Ringel

Deep neural networks (DNNs) are powerful tools for compressing and distilling information.

A self consistent theory of Gaussian Processes captures feature learning effects in finite CNNs

no code implementations • NeurIPS 2021 • Gadi Naveh, Zohar Ringel

Deep neural networks (DNNs) in the infinite width/channel limit have received much attention recently, as they provide a clear analytical window to deep learning via mappings to Gaussian Processes (GPs).

Gaussian Processes

Predicting the Outputs of Finite Networks Trained with Noisy Gradients

no code implementations • 28 Sep 2020 • Gadi Naveh, Oded Ben-David, Haim Sompolinsky, Zohar Ringel

A recent line of works studied wide deep neural networks (DNNs) by approximating them as Gaussian Processes (GPs).

Gaussian Processes Image Classification

Predicting the outputs of finite deep neural networks trained with noisy gradients

no code implementations • 2 Apr 2020 • Gadi Naveh, Oded Ben-David, Haim Sompolinsky, Zohar Ringel

A recent line of works studied wide deep neural networks (DNNs) by approximating them as Gaussian Processes (GPs).

Gaussian Processes Image Classification