no code implementations • 23 Jan 2024 • Elizaveta Demyanenko, Christoph Feinauer, Enrico M. Malatesta, Luca Saglietti
Recent works demonstrated the existence of a double-descent phenomenon for the generalization error of neural networks, where highly overparameterized models escape overfitting and achieve good test performance, at odds with the standard bias-variance trade-off described by statistical learning theory.
no code implementations • 18 May 2023 • Brandon Livio Annesi, Clarissa Lauditi, Carlo Lucibello, Enrico M. Malatesta, Gabriele Perugini, Fabrizio Pittorino, Luca Saglietti
Empirical studies on the landscape of neural networks have shown that low-energy configurations are often found in complex connected structures, where zero-energy paths between pairs of distant solutions can be constructed.
no code implementations • 2 Mar 2023 • Federica Gerace, Diego Doimo, Stefano Sarao Mannelli, Luca Saglietti, Alessandro Laio
The simplest transfer learning protocol is based on ``freezing" the feature-extractor layers of a network pre-trained on a data-rich source task, and then adapting only the last layers to a data-poor target task.
no code implementations • 31 May 2022 • Stefano Sarao Mannelli, Federica Gerace, Negar Rostamzadeh, Luca Saglietti
Then, we consider a novel mitigation strategy based on a matched inference approach, consisting in the introduction of coupled learning models.
no code implementations • 15 Jun 2021 • Luca Saglietti, Stefano Sarao Mannelli, Andrew Saxe
To study the former, we provide an exact description of the online learning setting, confirming the long-standing experimental observation that curricula can modestly speed up learning.
no code implementations • 9 Jun 2021 • Federica Gerace, Luca Saglietti, Stefano Sarao Mannelli, Andrew Saxe, Lenka Zdeborová
Transfer learning can significantly improve the sample efficiency of neural networks, by exploiting the relatedness between a data-scarce target task and a data-abundant source task.
no code implementations • 1 Dec 2020 • Luca Saglietti, Lenka Zdeborová
In recent years the empirical success of transfer learning with neural networks has stimulated an increasing interest in obtaining a theoretical understanding of its core properties.
no code implementations • 9 Dec 2019 • Hugo Cui, Luca Saglietti, Lenka Zdeborová
These large deviations then provide optimal achievable performance boundaries for any active learning algorithm.
no code implementations • 13 May 2019 • Luca Saglietti, Yue M. Lu, Carlo Lucibello
In Generalized Linear Estimation (GLE) problems, we seek to estimate a signal that is observed through a linear transform followed by a component-wise, possibly nonlinear and noisy, channel.
2 code implementations • NeurIPS 2018 • Francesco Paolo Casale, Adrian V. Dalca, Luca Saglietti, Jennifer Listgarten, Nicolo Fusi
In this work, we introduce a new model, the Gaussian Process (GP) Prior Variational Autoencoder (GPPVAE), to specifically address this issue.
no code implementations • 26 Oct 2017 • Carlo Baldassi, Federica Gerace, Hilbert J. Kappen, Carlo Lucibello, Luca Saglietti, Enzo Tartaglione, Riccardo Zecchina
Stochasticity and limited precision of synaptic weights in neural network models are key aspects of both biological and hardware modeling of learning processes.
no code implementations • 20 May 2016 • Carlo Baldassi, Christian Borgs, Jennifer Chayes, Alessandro Ingrosso, Carlo Lucibello, Luca Saglietti, Riccardo Zecchina
We define a novel measure, which we call the "robust ensemble" (RE), which suppresses trapping by isolated configurations and amplifies the role of these dense regions.
no code implementations • 12 Feb 2016 • Carlo Baldassi, Federica Gerace, Carlo Lucibello, Luca Saglietti, Riccardo Zecchina
Learning in neural networks poses peculiar challenges when using discretized rather then continuous synaptic states.
no code implementations • 18 Nov 2015 • Carlo Baldassi, Alessandro Ingrosso, Carlo Lucibello, Luca Saglietti, Riccardo Zecchina
We introduce a novel Entropy-driven Monte Carlo (EdMC) strategy to efficiently sample solutions of random Constraint Satisfaction Problems (CSPs).
no code implementations • 18 Sep 2015 • Carlo Baldassi, Alessandro Ingrosso, Carlo Lucibello, Luca Saglietti, Riccardo Zecchina
We also show that the dense regions are surprisingly accessible by simple learning protocols, and that these synaptic configurations are robust to perturbations and generalize better than typical solutions.