Network Pruning
213 papers with code • 5 benchmarks • 5 datasets
Network Pruning is a popular approach to reduce a heavy network to obtain a light-weight form by removing redundancy in the heavy network. In this approach, a complex over-parameterized network is first trained, then pruned based on come criterions, and finally fine-tuned to achieve comparable performance with reduced parameters.
Source: Ensemble Knowledge Distillation for Learning Improved and Efficient Networks
Benchmarks
These leaderboards are used to track progress in Network Pruning
| Trend | Dataset | Best Model | Paper | Code | Compare |
|---|---|---|---|---|---|
|
|||||
|
|||||
|
|||||
|
|||||
|
Libraries
Use these libraries to find Network Pruning models and implementations
CIFAR-10
ImageNet
CIFAR-100
MNIST
Netzschleuder
Latest papers
Auto-Train-Once: Controller Network Guided Automatic Network Pruning from Scratch
Current techniques for deep neural network (DNN) pruning often involve intricate multi-step processes that require domain-specific expertise, making their widespread adoption challenging.
Adversarial Fine-tuning of Compressed Neural Networks for Joint Improvement of Robustness and Efficiency
saintslab/pepr
•
•
We present experiments on two benchmark datasets showing that adversarial fine-tuning of compressed models can achieve robustness performance comparable to adversarially trained models, while also improving computational efficiency.
FALCON: FLOP-Aware Combinatorial Optimization for Neural Network Pruning
In this paper, we propose FALCON, a novel combinatorial-optimization-based framework for network pruning that jointly takes into account model accuracy (fidelity), FLOPs, and sparsity constraints.
What to Do When Your Discrete Optimization Is the Size of a Neural Network?
Oftentimes, machine learning applications using neural networks involve solving discrete optimization problems, such as in pruning, parameter-isolation-based continual learning and training of binary networks.
Less is KEN: a Universal and Simple Non-Parametric Pruning Algorithm for Large Language Models
itsmattei/ken
•
•
This approach maintains model performance while allowing storage of only the optimized subnetwork, leading to significant memory savings.
Fluctuation-based Adaptive Structured Pruning for Large Language Models
Retraining-free is important for LLMs' pruning methods.
Towards Higher Ranks via Adversarial Weight Pruning
huawei-noah/Efficient-Computing
•
•
NeurIPS 2023
To this end, we propose a Rank-based PruninG (RPG) method to maintain the ranks of sparse weights in an adversarial manner.
LightGaussian: Unbounded 3D Gaussian Compression with 15x Reduction and 200+ FPS
Recent advancements in real-time neural rendering using point-based techniques have paved the way for the widespread adoption of 3D representations.
Filter-Pruning of Lightweight Face Detectors Using a Geometric Median Criterion
idt-iti/lightweight-face-detector-pruning
•
•
Face detectors are becoming a crucial component of many applications, including surveillance, that often have to run on edge devices with limited processing power and memory.
Neural Network Pruning by Gradient Descent
The rapid increase in the parameters of deep learning models has led to significant costs, challenging computational efficiency and model interpretability.
