TASK	DATASET	MODEL	METRIC NAME	METRIC VALUE	GLOBAL RANK
Image Classification	ImageNet	KDforAA (EfficientNet-B7)	Top 1 Accuracy	85.5%	# 212
Image Classification	ImageNet	KDforAA (EfficientNet-B7)	Number of params	66M	# 777
Image Classification	ImageNet	KDforAA (EfficientNet-B8)	Top 1 Accuracy	85.8%	# 187
Image Classification	ImageNet	KDforAA (EfficientNet-B8)	Number of params	88M	# 832
Image Classification	ImageNet	KDforAA (EfficientNet-B8)	Hardware Burden	None	# 1
Image Classification	ImageNet	KDforAA (EfficientNet-B8)	Operations per network pass	None	# 1

Badge	Markdown
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/circumventing-outliers-of-autoaugment-with/image-classification-on-imagenet)](https://paperswithcode.com/sota/image-classification-on-imagenet?p=circumventing-outliers-of-autoaugment-with)`

Circumventing Outliers of AutoAugment with Knowledge Distillation

ECCV 2020 · Longhui Wei, An Xiao, Lingxi Xie, Xin Chen, Xiaopeng Zhang, Qi Tian ·

AutoAugment has been a powerful algorithm that improves the accuracy of many vision tasks, yet it is sensitive to the operator space as well as hyper-parameters, and an improper setting may degenerate network optimization. This paper delves deep into the working mechanism, and reveals that AutoAugment may remove part of discriminative information from the training image and so insisting on the ground-truth label is no longer the best option. To relieve the inaccuracy of supervision, we make use of knowledge distillation that refers to the output of a teacher model to guide network training. Experiments are performed in standard image classification benchmarks, and demonstrate the effectiveness of our approach in suppressing noise of data augmentation and stabilizing training. Upon the cooperation of knowledge distillation and AutoAugment, we claim the new state-of-the-art on ImageNet classification with a top-1 accuracy of 85.8%.

PDF Abstract ECCV 2020 PDF ECCV 2020 Abstract

Code

Add Remove Mark official

chenyang4/resmooth

Tasks

Add Remove

Data Augmentation

General Classification

Image Classification

Knowledge Distillation

Datasets

CIFAR-10

ImageNet

CIFAR-100

Results from the Paper

Edit

Ranked #185 on Image Classification on ImageNet

Get a GitHub badge

Task	Dataset	Model	Metric Name	Metric Value	Global Rank	Benchmark
Image Classification	ImageNet	KDforAA (EfficientNet-B7)	Top 1 Accuracy	85.5%	# 212	Compare
Image Classification	ImageNet	KDforAA (EfficientNet-B7)	Number of params	66M	# 777	Compare
Image Classification	ImageNet	KDforAA (EfficientNet-B8)	Top 1 Accuracy	85.8%	# 187	Compare
			Number of params	88M	# 832	Compare
			Hardware Burden	None	# 1	Compare
			Operations per network pass	None	# 1	Compare

Methods

Add Remove

1x1 Convolution • AutoAugment • Average Pooling • Batch Normalization • Convolution • Dense Connections • Depthwise Convolution • Depthwise Separable Convolution • Dropout • EfficientNet • FixRes • Inverted Residual Block • Knowledge Distillation • LSTM • Pointwise Convolution • RandAugment • ReLU • RMSProp • ShakeDrop • Sigmoid Activation • Squeeze-and-Excitation Block • Swish • Tanh Activation • Weight Decay

Edit Social Preview

Circumventing Outliers of AutoAugment with Knowledge Distillation

Code Edit Add Remove Mark official

Tasks Edit Add Remove

Datasets Edit

Results from the Paper Edit

Methods Edit Add Remove

Code

Add Remove Mark official

Tasks

Add Remove

Datasets

Results from the Paper

Edit

Methods

Add Remove