Zero-padded Shortcut Connection

Introduced by Han et al. in Deep Pyramidal Residual Networks

A Zero-padded Shortcut Connection is a type of residual connection used in the PyramidNet architecture. For PyramidNets, identity mapping alone cannot be used for a shortcut because the feature map dimension differs among individual residual units. Therefore, only a zero-padded shortcut or projection shortcut can be used for all the residual units. However, a projection shortcut can hamper information propagation and lead to optimization problems, especially for very deep networks. On the other hand, the zero-padded shortcut avoids the overfitting problem because no additional parameters exist.

Source: Deep Pyramidal Residual Networks

Latest Papers

PAPER DATE
Averaging Weights Leads to Wider Optima and Better Generalization
| Pavel IzmailovDmitrii PodoprikhinTimur GaripovDmitry VetrovAndrew Gordon Wilson
2018-03-14
ShakeDrop Regularization for Deep Residual Learning
| Yoshihiro YamadaMasakazu IwamuraTakuya AkibaKoichi Kise
2018-02-07
ShakeDrop regularization
Yoshihiro YamadaMasakazu IwamuraKoichi Kise
2018-01-01
Improving Generalization Performance by Switching from Adam to SGD
| Nitish Shirish KeskarRichard Socher
2017-12-20
Pyramidal RoR for Image Classification
Ke ZhangLiru GuoCe GaoZhenbing Zhao
2017-10-01
Deep Pyramidal Residual Networks
Dongyoon HanJiwhan KimJunmo Kim
2017-07-01
Deep Pyramidal Residual Networks with Separated Stochastic Depth
Yoshihiro YamadaMasakazu IwamuraKoichi Kise
2016-12-05
Deep Pyramidal Residual Networks
| Dongyoon HanJiwhan KimJunmo Kim
2016-10-10

Tasks

TASK PAPERS SHARE
Image Classification 4 66.67%
Language Modelling 1 16.67%
Object Recognition 1 16.67%

Components

COMPONENT TYPE
🤖 No Components Found You can add them if they exist; e.g. Mask R-CNN uses RoIAlign

Categories