Network Pruning
213 papers with code • 5 benchmarks • 5 datasets
Network Pruning is a popular approach to reduce a heavy network to obtain a light-weight form by removing redundancy in the heavy network. In this approach, a complex over-parameterized network is first trained, then pruned based on come criterions, and finally fine-tuned to achieve comparable performance with reduced parameters.
Source: Ensemble Knowledge Distillation for Learning Improved and Efficient Networks
Benchmarks
These leaderboards are used to track progress in Network Pruning
| Trend | Dataset | Best Model | Paper | Code | Compare |
|---|---|---|---|---|---|
|
|||||
|
|||||
|
|||||
|
|||||
|
Libraries
Use these libraries to find Network Pruning models and implementations
CIFAR-10
ImageNet
CIFAR-100
MNIST
Netzschleuder
Latest papers with no code
GD doesn't make the cut: Three ways that non-differentiability affects neural network training
This paper investigates the distinctions between gradient methods applied to non-differentiable functions (NGDMs) and classical gradient descents (GDs) designed for differentiable functions.
Block Pruning for Enhanced Efficiency in Convolutional Neural Networks
This paper presents a novel approach to network pruning, targeting block pruning in deep neural networks for edge computing environments.
Picking the Underused Heads: A Network Pruning Perspective of Attention Head Selection for Fusing Dialogue Coreference Information
In this work, we investigate the attention head selection and manipulation strategy for feature injection from a network pruning perspective, and conduct a case study on dialogue summarization.
Neural Architecture Codesign for Fast Bragg Peak Analysis
We develop an automated pipeline to streamline neural architecture codesign for fast, real-time Bragg peak analysis in high-energy diffraction microscopy.
Accelerating Convolutional Neural Network Pruning via Spatial Aura Entropy
We propose a novel method to improve MI computation for CNN pruning, using the spatial aura entropy.
An End-to-End Network Pruning Pipeline with Sparsity Enforcement
Neural networks have emerged as a powerful tool for solving complex tasks across various domains, but their increasing size and computational requirements have posed significant challenges in deploying them on resource-constrained devices.
Robustness-Reinforced Knowledge Distillation with Correlation Distance and Network Pruning
The improvement in the performance of efficient and lightweight models (i. e., the student model) is achieved through knowledge distillation (KD), which involves transferring knowledge from more complex models (i. e., the teacher model).
Mechanistically analyzing the effects of fine-tuning on procedurally defined tasks
Fine-tuning large pre-trained models has become the de facto strategy for developing both task-specific and general-purpose machine learning systems, including developing models that are safe to deploy.
Data Augmentations in Deep Weight Spaces
Learning in weight spaces, where neural networks process the weights of other deep neural networks, has emerged as a promising research direction with applications in various fields, from analyzing and editing neural fields and implicit neural representations, to network pruning and quantization.
Criticality-Guided Efficient Pruning in Spiking Neural Networks Inspired by Critical Brain Hypothesis
Firstly, we propose a low-cost metric for the criticality in SNNs.
