Squeeze-and-Excitation Block

Introduced by Hu et al. in Squeeze-and-Excitation Networks

The Squeeze-and-Excitation Block is an architectural unit designed to improve the representational power of a network by enabling it to perform dynamic channel-wise feature recalibration. The process is:

  • The block has a convolutional block as an input.
  • Each channel is "squeezed" into a single numeric value using average pooling.
  • A dense layer followed by a ReLU adds non-linearity and output channel complexity is reduced by a ratio.
  • Another dense layer followed by a sigmoid gives each channel a smooth gating function.
  • Finally, we weight each feature map of the convolutional block based on the side network; the "excitation".
Source: Squeeze-and-Excitation Networks

Latest Papers

PAPER DATE
WeightNet: Revisiting the Design Space of Weight Networks
| Ningning MaXiangyu ZhangJiawei HuangJian Sun
2020-07-23
PareCO: Pareto-aware Channel Optimization for Slimmable Neural Networks
Ting-Wu ChinAri S. MorcosDiana Marculescu
2020-07-23
EfficientHRNet: Efficient Scaling for Lightweight High-Resolution Multi-Person Pose Estimation
Christopher NeffAneri ShethSteven FurgursonHamed Tabkhi
2020-07-16
To filter prune, or to layer prune, that is the question
Sara ElkerdawyMostafa ElhoushiAbhineet SinghHong ZhangNilanjan Ray
2020-07-11
NVAE: A Deep Hierarchical Variational Autoencoder
Arash VahdatJan Kautz
2020-07-08
Fine-Grained Stochastic Architecture Search
Shraman Ray ChaudhuriElad EbanHanhan LiMax MorozYair Movshovitz-Attias
2020-06-17
Improving accuracy and speeding up Document Image Classification through parallel systems
Javier FerrandoJuan Luis DominguezJordi TorresRaul GarciaDavid GarciaDaniel GarridoJordi CortadaMateo Valero
2020-06-16
A heterogeneous branch and multi-level classification network for person re-identification
Jiabao WangYang LiYangshuo ZhangZhuang MiaoRui Zhang
2020-06-02
Residual Squeeze-and-Excitation Network for Fast Image Deraining
Jun FuJianfeng XuKazuyuki TasakaZhibo Chen
2020-06-01
Deep Learning based Diagnosis of COVID-19 usingChest CT-scan Images
| Talha AnwarSeemab Zakir
2020-05-20
Comparison and Benchmarking of AI Models and Frameworks on Mobile Devices
Chunjie LuoXiwen HeJianfeng ZhanLei WangWanling GaoJiahui Dai
2020-05-07
AI Augmentation of Radiologist Performance in Distinguishing COVID-19 from Pneumonia of Other Etiology on Chest CT
| Harrison X. BaiRobin WangZeng XiongBen HsiehKen ChangKasey HalseyThi My Linh TranJi Whae ChoiDong-Cui WangLin-Bo ShiJi MeiXiao-Long JiangIan PanQiu-Hua ZengPing-Feng HuYi-Hui LiFei-Xian FuRaymond Y. HuangRonnie SebroQi-Zhi YuMichael K. AtalayWei-Hua Liao
2020-04-27
LSQ+: Improving low-bit quantization through learnable offsets and better initialization
| Yash BhalgatJinwon LeeMarkus NagelTijmen BlankevoortNojun Kwak
2020-04-20
DriftNet: Aggressive Driving Behavior Classification using 3D EfficientNet Architecture
Alam NoorBilel BenjdiraAdel AmmarAnis Koubaa
2020-04-18
Video Face Manipulation Detection Through Ensemble of CNNs
| Nicolò BonettiniEdoardo Daniele CannasSara MandelliLuca BondiPaolo BestaginiStefano Tubaro
2020-04-16
An Evaluation of DNN Architectures for Page Segmentation of Historical Newspapers
| Bernhard LieblManuel Burghardt
2020-04-15
Towards an Effective and Efficient Deep Learning Model for COVID-19 Patterns Detection in X-ray Images
| Eduardo LuzPedro Lopes SilvaRodrigo SilvaLudmila SilvaGladston MoreiraDavid Menotti
2020-04-12
Analysis on DeepLabV3+ Performance for Automatic Steel Defects Detection
Zheng NieJiachen XuShengchang Zhang
2020-04-09
AutoToon: Automatic Geometric Warping for Face Cartoon Generation
Julia GongYannick Hold-GeoffroyJingwan Lu
2020-04-06
Evolving Normalization-Activation Layers
| Hanxiao LiuAndrew BrockKaren SimonyanQuoc V. Le
2020-04-06
NBDT: Neural-Backed Decision Trees
| Alvin WanLisa DunlapDaniel HoJihan YinScott LeeHenry JinSuzanne PetrykSarah Adel BargalJoseph E. Gonzalez
2020-04-01
Designing Network Design Spaces
| Ilija RadosavovicRaj Prateek KosarajuRoss GirshickKaiming HePiotr Dollár
2020-03-30
TResNet: High Performance GPU-Dedicated Architecture
| Tal RidnikHussam LawenAsaf NoyItamar FriedmanEmanuel Ben BaruchGilad Sharir
2020-03-30
Circumventing Outliers of AutoAugment with Knowledge Distillation
Longhui WeiAn XiaoLingxi XieXin ChenXiaopeng ZhangQi Tian
2020-03-25
GreedyNAS: Towards Fast One-Shot NAS with Greedy Supernet
Shan YouTao HuangMingmin YangFei WangChen QianChangshui Zhang
2020-03-25
Multi-Plateau Ensemble for Endoscopic Artefact Segmentation and Detection
| Suyog JadhavUdbhav BambaArnav ChavanRishabh TiwariAryan Raj
2020-03-23
Meta Pseudo Labels
Hieu PhamQizhe XieZihang DaiQuoc V. Le
2020-03-23
Fixing the train-test resolution discrepancy: FixEfficientNet
| Hugo TouvronAndrea VedaldiMatthijs DouzeHervé Jégou
2020-03-18
Gimme Signals: Discriminative signal encoding for multimodal activity recognition
| Raphael MemmesheimerNick TheisenDietrich Paulus
2020-03-13
Learned Threshold Pruning
Kambiz AzarianYash BhalgatJinwon LeeTijmen Blankevoort
2020-02-28
MaxUp: A Simple Way to Improve Generalization of Neural Network Training
Chengyue GongTongzheng RenMao YeQiang Liu
2020-02-20
Skip Connections Matter: On the Transferability of Adversarial Examples Generated with ResNets
Dongxian WuYisen WangShu-Tao XiaJames BaileyXingjun Ma
2020-02-14
A closer look at network resolution for efficient network design
Anonymous
2020-01-01
Attention-Based Face AntiSpoofing of RGB Images, using a Minimal End-2-End Neural Network
Ali GhofraniRahil Mahdian ToroghiSeyed Mojtaba Tabatabaie
2019-12-18
CSPNet: A New Backbone that can Enhance Learning Capability of CNN
| Chien-Yao WangHong-Yuan Mark LiaoI-Hau YehYueh-Hua WuPing-Yang ChenJun-Wei Hsieh
2019-11-27
GhostNet: More Features from Cheap Operations
| Kai HanYunhe WangQi TianJianyuan GuoChunjing XuChang Xu
2019-11-27
Structured Multi-Hashing for Model Compression
Elad EbanYair Movshovitz-AttiasHao WuMark SandlerAndrew PoonYerlan IdelbayevMiguel A. Carreira-Perpinan
2019-11-25
Panoptic-DeepLab: A Simple, Strong, and Fast Baseline for Bottom-Up Panoptic Segmentation
| Bowen ChengMaxwell D. CollinsYukun ZhuTing LiuThomas S. HuangHartwig AdamLiang-Chieh Chen
2019-11-22
Adversarial Examples Improve Image Recognition
| Cihang XieMingxing TanBoqing GongJiang WangAlan YuilleQuoc V. Le
2019-11-21
Fast Sparse ConvNets
| Erich ElsenMarat DukhanTrevor GaleKaren Simonyan
2019-11-21
EfficientDet: Scalable and Efficient Object Detection
| Mingxing TanRuoming PangQuoc V. Le
2019-11-20
Experimental Exploration of Compact Convolutional Neural Network Architectures for Non-temporal Real-time Fire Detection
Ganesh Samarth C. A.Neelanjan BhowmikToby P. Breckon
2019-11-20
Self-training with Noisy Student improves ImageNet classification
| Qizhe XieMinh-Thang LuongEduard HovyQuoc V. Le
2019-11-11
Identification of primary angle-closure on AS-OCT images with Convolutional Neural Networks
Chenglang YuanCheng BianHongjian KangShu LiangKai MaYefeng Zheng
2019-10-23
ICPS-net: An End-to-End RGB-based Indoor Camera Positioning System using deep convolutional neural networks
Ali GhofraniRahil Mahdian ToroghiSayed Mojtaba Tabatabaie
2019-10-14
ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks
| Qilong WangBanggu WuPengfei ZhuPeihua LiWangmeng ZuoQinghua Hu
2019-10-08
SRM: A Style-Based Recalibration Module for Convolutional Neural Networks
HyunJae Lee Hyo-Eun Kim Hyeonseob Nam
2019-10-01
RandAugment: Practical automated data augmentation with a reduced search space
| Ekin D. CubukBarret ZophJonathon ShlensQuoc V. Le
2019-09-30
MutualNet: Adaptive ConvNet via Mutual Learning from Network Width and Resolution
| Taojiannan YangSijie ZhuChen ChenShen YanMi ZhangAndrew Willis
2019-09-27
Street Crossing Aid Using Light-weight CNNs for the Visually Impaired
| Samuel YuHeon LeeJung Hoon Kim
2019-09-14
Mish: A Self Regularized Non-Monotonic Neural Activation Function
| Diganta Misra
2019-08-23
MoGA: Searching Beyond MobileNetV3
| Xiangxiang ChuBo ZhangRuijun Xu
2019-08-04
MixConv: Mixed Depthwise Convolutional Kernels
| Mingxing TanQuoc V. Le
2019-07-22
Neural Epitome Search for Architecture-Agnostic Network Compression
Daquan ZhouXiaojie JinQibin HouKaixin WangJianchao YangJiashi Feng
2019-07-12
Butterfly Transform: An Efficient FFT Based Neural Architecture Design
| Keivan Alizadeh VahidAnish PrabhuAli FarhadiMohammad Rastegari
2019-06-05
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
| Mingxing TanQuoc V. Le
2019-05-28
Advancements in Image Classification using Convolutional Neural Network
Farhana SultanaA. SufianParamartha Dutta
2019-05-08
Searching for MobileNetV3
| Andrew HowardMark SandlerGrace ChuLiang-Chieh ChenBo ChenMingxing TanWeijun WangYukun ZhuRuoming PangVijay VasudevanQuoc V. LeHartwig Adam
2019-05-06
GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyond
| Yue CaoJiarui XuStephen LinFangyun WeiHan Hu
2019-04-25
Attention Augmented Convolutional Networks
| Irwan BelloBarret ZophAshish VaswaniJonathon ShlensQuoc V. Le
2019-04-22
Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks with Octave Convolution
| Yunpeng ChenHaoqi FanBing XuZhicheng YanYannis KalantidisMarcus RohrbachShuicheng YanJiashi Feng
2019-04-10
CondConv: Conditionally Parameterized Convolutions for Efficient Inference
| Brandon YangGabriel BenderQuoc V. LeJiquan Ngiam
2019-04-10
AutoSlim: Towards One-Shot Architecture Search for Channel Numbers
| Jiahui YuThomas Huang
2019-03-27
Hybrid Task Cascade for Instance Segmentation
| Kai ChenJiangmiao PangJiaqi WangYu XiongXiaoxiao LiShuyang SunWansen FengZiwei LiuJianping ShiWanli OuyangChen Change LoyDahua Lin
2019-01-22
3D Object Detection Using Scale Invariant and Feature Reweighting Networks
Xin ZhaoZhe LiuRuolan HuKaiqi Huang
2019-01-08
Channel Locality Block: A Variant of Squeeze-and-Excitation
Huayu Li
2019-01-06
ChamNet: Towards Efficient Network Design through Platform-Aware Model Adaptation
| Xiaoliang DaiPeizhao ZhangBichen WuHongxu YinFei SunYanghan WangMarat DukhanYunqing HuYiming WuYangqing JiaPeter VajdaMatt UyttendaeleNiraj K. Jha
2018-12-21
Graph-Based Global Reasoning Networks
| Yunpeng ChenMarcus RohrbachZhicheng YanShuicheng YanJiashi FengYannis Kalantidis
2018-11-30
Learning Better Features for Face Detection with Feature Fusion and Segmentation Supervision
Wanxin TianZixuan WangHaifeng ShenWeihong DengYiping MengBinghui ChenXiubao ZhangYuan ZhaoXiehe Huang
2018-11-20
Skin Lesion Diagnosis using Ensembles, Unscaled Multi-Crop Evaluation and Loss Weighting
| Nils GessertThilo SentkerFrederic MadestaRüdiger SchmitzHelge KniepIvo BaltruschatRené WernerAlexander Schlaefer
2018-08-05
MnasNet: Platform-Aware Neural Architecture Search for Mobile
| Mingxing TanBo ChenRuoming PangVijay VasudevanMark SandlerAndrew HowardQuoc V. Le
2018-07-31
ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design
| Ningning MaXiangyu ZhangHai-Tao ZhengJian Sun
2018-07-30
Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net
| Xingang PanPing LuoJianping ShiXiaoou Tang
2018-07-25
CBAM: Convolutional Block Attention Module
| Sanghyun WooJongchan ParkJoon-Young LeeIn So Kweon
2018-07-17
Multivariate LSTM-FCNs for Time Series Classification
| Fazle KarimSomshubra MajumdarHoushang DarabiSamuel Harford
2018-01-14
Improving Generalization Performance by Switching from Adam to SGD
| Nitish Shirish KeskarRichard Socher
2017-12-20
AOGNets: Compositional Grammatical Architectures for Deep Learning
| Xilai LiXi SongTianfu Wu
2017-11-15
Squeeze-and-Excitation Networks
| Jie HuLi ShenSamuel AlbanieGang SunEnhua Wu
2017-09-05

Categories