Browse SoTA > Methodology > Model Compression

Model Compression

71 papers with code ยท Methodology

Model Compression is an actively pursued area of research over the last few years with the goal of deploying state-of-the-art deep networks in low-power and resource limited devices without significant drop in accuracy. Parameter pruning, low-rank factorization and weight quantization are some of the proposed methods to compress the size of deep networks.

Source: KD-MRI: A knowledge distillation framework for image reconstruction and image restoration in MRI workflow

Benchmarks

You can find evaluation results in the subtasks. You can also submitting evaluation metrics for this task.

Latest papers without code

Compressing Deep Neural Networks via Layer Fusion

29 Jul 2020

This paper proposes \textit{layer fusion} - a model compression technique that discovers which weights to combine and then fuses weights of similar fully-connected, convolutional and attention layers.

LANGUAGE MODELLING MODEL COMPRESSION

ALF: Autoencoder-based Low-rank Filter-sharing for Efficient Convolutional Neural Networks

27 Jul 2020

Model compression techniques, such as pruning, are emphasized among other optimization methods for solving this problem.

MODEL COMPRESSION

Achieving Real-Time Execution of 3D Convolutional Neural Networks on Mobile Devices

20 Jul 2020

The vanilla sparsity removes whole kernel groups, while KGS sparsity is a more fine-grained structured sparsity that enjoys higher flexibility while exploiting full on-device parallelism.

CODE GENERATION MODEL COMPRESSION

FTRANS: Energy-Efficient Acceleration of Transformers using FPGA

16 Jul 2020

In natural language processing (NLP), the "Transformer" architecture was proposed as the first transduction model replying entirely on self-attention mechanisms without using sequence-aligned recurrent neural networks (RNNs) or convolution, and it achieved significant improvements for sequence to sequence tasks.

MODEL COMPRESSION

Representation Transfer by Optimal Transport

13 Jul 2020

Deep learning currently provides the best representations of complex objects for a wide variety of tasks.

MODEL COMPRESSION TRANSFER LEARNING

Learning to Prune Deep Neural Networks via Reinforcement Learning

9 Jul 2020

PuRL achieves more than 80% sparsity on the ResNet-50 model while retaining a Top-1 accuracy of 75. 37% on the ImageNet dataset.

MODEL COMPRESSION

Soft Labeling Affects Out-of-Distribution Detection of Deep Neural Networks

7 Jul 2020

Soft labeling becomes a common output regularization for generalization and model compression of deep neural networks.

MODEL COMPRESSION OUT-OF-DISTRIBUTION DETECTION

Self-Supervised GAN Compression

ICLR 2020

Deep learning's success has led to larger and larger models to handle more and more complex tasks; trained models can contain millions of parameters.

IMAGE CLASSIFICATION MODEL COMPRESSION

Knowledge Distillation Beyond Model Compression

3 Jul 2020

Knowledge distillation (KD) is commonly deemed as an effective model compression technique in which a compact model (student) is trained under the supervision of a larger pretrained model or an ensemble of models (teacher).

MODEL COMPRESSION

Channel Compression: Rethinking Information Redundancy among Channels in CNN Architecture

2 Jul 2020

Aiming at channel compression, a novel convolutional construction named compact convolution is proposed to embrace the progress in spatial convolution, channel grouping and pooling operation.

ACOUSTIC SCENE CLASSIFICATION IMAGE CLASSIFICATION MODEL COMPRESSION SCENE CLASSIFICATION SOUND EVENT DETECTION