Browse SoTA > Methodology > Model Compression

# Model Compression Edit

71 papers with code · Methodology

Model Compression is an actively pursued area of research over the last few years with the goal of deploying state-of-the-art deep networks in low-power and resource limited devices without significant drop in accuracy. Parameter pruning, low-rank factorization and weight quantization are some of the proposed methods to compress the size of deep networks.

You can find evaluation results in the subtasks. You can also submitting evaluation metrics for this task.

# Paying more attention to snapshots of Iterative Pruning: Improving Model Compression via Ensemble Distillation

20 Jun 2020lehduong/ginp

In this work, we show that strong ensembles can be constructed from snapshots of iterative pruning, which achieve competitive performance and vary in network structure.

4
20 Jun 2020

# Improving Post Training Neural Quantization: Layer-wise Calibration and Integer Programming

14 Jun 2020itayhubara/CalibTIP

Most of the literature on neural network quantization requires some training of the quantized model (fine-tuning).

9
14 Jun 2020

# Knowledge Distillation Meets Self-Supervision

12 Jun 2020xuguodong03/SSKD

Knowledge distillation, which involves extracting the "dark knowledge" from a teacher network to guide the learning of a student network, has emerged as an important technique for model compression and transfer learning.

80
12 Jun 2020

# VecQ: Minimal Loss DNN Model Compression With Vectorized Weight Quantization

18 May 2020GongCheng1919/VecQ

Quantization has been proven to be an effective method for reducing the computing and/or storage cost of DNNs.

1
18 May 2020

# MicroNet for Efficient Language Modeling

16 May 2020mit-han-lab/neurips-micronet

In this paper, we provide the winning solution to the NeurIPS 2019 MicroNet Challenge in the language modeling track.

15
16 May 2020

# A flexible, extensible software framework for model compression based on the LC algorithm

15 May 2020UCMerced-ML/LC-model-compression

We propose a software framework based on the ideas of the Learning-Compression (LC) algorithm, that allows a user to compress a neural network or other machine learning model using different compression schemes with minimal effort.

14
15 May 2020

# Training with Quantization Noise for Extreme Model Compression

15 Apr 2020pytorch/fairseq

A standard solution is to train networks with Quantization Aware Training, where the weights are quantized during training and the gradients approximated with the Straight-Through Estimator.

8,567
15 Apr 2020

# KD-MRI: A knowledge distillation framework for image reconstruction and image restoration in MRI workflow

11 Apr 2020Bala93/KD-MRI

In our work, we propose a knowledge distillation (KD) framework for the image to image problems in the MRI workflow in order to develop compact, low-parameter models without a significant drop in performance.

3
11 Apr 2020

# Orthant Based Proximal Stochastic Gradient Method for $\ell_1$-Regularized Optimization

7 Apr 2020tianyic/obproxsg

Sparsity-inducing regularization problems are ubiquitous in machine learning applications, ranging from feature selection to model compression.

3
07 Apr 2020