Search Results for author:
Found 6 papers, 5 papers with code
PERP: Rethinking the Prune-Retrain Paradigm in the Era of LLMs
Neural Networks can be efficiently compressed through pruning, significantly reducing storage and computational demands while maintaining predictive performance.
Sparse Model Soups: A Recipe for Improved Pruning via Model Averaging
Model soups (Wortsman et al., 2022) enhance generalization and out-of-distribution (OOD) performance by averaging the parameters of multiple models into a single one, without increasing inference time.
Compression-aware Training of Neural Networks using Frank-Wolfe
Many existing Neural Network pruning approaches rely on either retraining or inducing a strong bias in order to converge to a sparse solution throughout training.
How I Learned to Stop Worrying and Love Retraining
Many Neural Network Pruning approaches consist of several iterative training and pruning steps, seemingly losing a significant amount of their performance after pruning and then recovering it in the subsequent retraining phase.
Deep Neural Network Training with Frank-Wolfe
In particular, we show the general feasibility of training Neural Networks whose parameters are constrained by a convex feasible region using Frank-Wolfe algorithms and compare different stochastic variants.
Projection-Free Adaptive Gradients for Large-Scale Optimization
The complexity in large-scale optimization can lie in both handling the objective function and handling the constraint set.
