Browse SoTA > Methodology > Model Compression

Model Compression

71 papers with code · Methodology

Model Compression is an actively pursued area of research over the last few years with the goal of deploying state-of-the-art deep networks in low-power and resource limited devices without significant drop in accuracy. Parameter pruning, low-rank factorization and weight quantization are some of the proposed methods to compress the size of deep networks.

Source: KD-MRI: A knowledge distillation framework for image reconstruction and image restoration in MRI workflow

Benchmarks

You can find evaluation results in the subtasks. You can also submitting evaluation metrics for this task.

Latest papers with code

Paying more attention to snapshots of Iterative Pruning: Improving Model Compression via Ensemble Distillation

20 Jun 2020lehduong/ginp

In this work, we show that strong ensembles can be constructed from snapshots of iterative pruning, which achieve competitive performance and vary in network structure.

IMAGE CLASSIFICATION MODEL COMPRESSION NETWORK PRUNING

4
20 Jun 2020

Improving Post Training Neural Quantization: Layer-wise Calibration and Integer Programming

14 Jun 2020itayhubara/CalibTIP

Most of the literature on neural network quantization requires some training of the quantized model (fine-tuning).

MODEL COMPRESSION QUANTIZATION

9
14 Jun 2020

Knowledge Distillation Meets Self-Supervision

12 Jun 2020xuguodong03/SSKD

Knowledge distillation, which involves extracting the "dark knowledge" from a teacher network to guide the learning of a student network, has emerged as an important technique for model compression and transfer learning.

CONTRASTIVE LEARNING MODEL COMPRESSION TRANSFER LEARNING

80
12 Jun 2020

VecQ: Minimal Loss DNN Model Compression With Vectorized Weight Quantization

18 May 2020GongCheng1919/VecQ

Quantization has been proven to be an effective method for reducing the computing and/or storage cost of DNNs.

MODEL COMPRESSION OBJECT DETECTION QUANTIZATION

1
18 May 2020

MicroNet for Efficient Language Modeling

16 May 2020mit-han-lab/neurips-micronet

In this paper, we provide the winning solution to the NeurIPS 2019 MicroNet Challenge in the language modeling track.

LANGUAGE MODELLING MODEL COMPRESSION NETWORK PRUNING QUANTIZATION

15
16 May 2020

A flexible, extensible software framework for model compression based on the LC algorithm

15 May 2020UCMerced-ML/LC-model-compression

We propose a software framework based on the ideas of the Learning-Compression (LC) algorithm, that allows a user to compress a neural network or other machine learning model using different compression schemes with minimal effort.

MODEL COMPRESSION QUANTIZATION

14
15 May 2020

Training with Quantization Noise for Extreme Model Compression

15 Apr 2020pytorch/fairseq

A standard solution is to train networks with Quantization Aware Training, where the weights are quantized during training and the gradients approximated with the Straight-Through Estimator.

IMAGE CLASSIFICATION MODEL COMPRESSION QUANTIZATION

8,567
15 Apr 2020

KD-MRI: A knowledge distillation framework for image reconstruction and image restoration in MRI workflow

11 Apr 2020Bala93/KD-MRI

In our work, we propose a knowledge distillation (KD) framework for the image to image problems in the MRI workflow in order to develop compact, low-parameter models without a significant drop in performance.

IMAGE RECONSTRUCTION IMAGE RESTORATION MODEL COMPRESSION SUPER-RESOLUTION

3
11 Apr 2020

Orthant Based Proximal Stochastic Gradient Method for $\ell_1$-Regularized Optimization

7 Apr 2020tianyic/obproxsg

Sparsity-inducing regularization problems are ubiquitous in machine learning applications, ranging from feature selection to model compression.

FEATURE SELECTION MODEL COMPRESSION

3
07 Apr 2020

A Survey on Contextual Embeddings

16 Mar 2020zhpmatrix/PaperReading

Contextual embeddings, such as ELMo and BERT, move beyond global word representations like Word2Vec and achieve ground-breaking performance on a wide range of natural language processing tasks.

MODEL COMPRESSION

47
16 Mar 2020