Spatial Attention-Guided Mask

Introduced by Lee et al. in CenterMask : Real-Time Anchor-Free Instance Segmentation

A Spatial Attention-Guided Mask is a module for instance segmentation that predicts a segmentation mask on each detected box with a spatial attention map that helps to focus on informative pixels and suppress noise. The goal is to guide the mask head for spotlighting meaningful pixels and repressing uninformative ones.

Once features inside the predicted RoIs are extracted by RoIAlign with 14×14 resolution, those features are fed into four conv layers and the spatial attention module (SAM) sequentially. To exploit the spatial attention map $A_{sag}\left(X_{i}\right) \in \mathcal{R}^{1\times{W}\times{H}}$ as a feature descriptor given input feature map $X_{i} \in \mathcal{R}^{C×W×H}$, the SAM first generates pooled features $P_{avg}, P_{max} \in \mathcal{R}^{1\times{W}\times{H}}$ by both average and max pooling operations respectively along the channel axis and aggregates them via concatenation. Then it is followed by a 3 × 3 conv layer and normalized by the sigmoid function. The computation process is summarized as follow:

$$ A_{sag}\left(X_{i}\right) = \sigma\left(F_{3\times{3}}(P_{max} \cdot P_{avg})\right) $$

where $\sigma$ denotes the sigmoid function, $F_{3\times{3}}$ is 3 × 3 conv layer and $\cdot$ represents the concatenate operation. Finally, the attention guided feature map $X_{sag} ∈ \mathcal{R}^{C\times{W}\times{H}}$ is computed as:

$$ X_{sag} = A_{sag}\left(X_{i}\right) \otimes X_{i} $$

where ⊗ denotes element-wise multiplication. After then, a 2 × 2 deconv upsamples the spatially attended feature map to 28 × 28 resolution. Lastly, a 1 × 1 conv is applied for predicting class-specific masks.

Source: CenterMask : Real-Time Anchor-Free Instance Segmentation

Latest Papers

CenterMask: single shot instance segmentation with point representation
Yuqing WangZhaoliang XuHao ShenBaoshan ChengLirong Yang
SA-UNet: Spatial Attention U-Net for Retinal Vessel Segmentation
| Changlu GuoMárton SzemenyeiYugen YiWenle WangBuer ChenChangqi Fan
Learning Oracle Attention for High-fidelity Face Completion
Tong ZhouChangxing DingShaowen LinXinchao WangDaCheng Tao
Context-Aware Domain Adaptation in Semantic Segmentation
Jinyu YangWeizhi AnChaochao YanPeilin ZhaoJunzhou Huang
CenterMask : Real-Time Anchor-Free Instance Segmentation
| Youngwan LeeJongyoul Park