Medical Transformer: Gated Axial-Attention for Medical Image Segmentation

21 Feb 2021  ·  Jeya Maria Jose Valanarasu, Poojan Oza, Ilker Hacihaliloglu, Vishal M. Patel ·

Over the past decade, Deep Convolutional Neural Networks have been widely adopted for medical image segmentation and shown to achieve adequate performance. However, due to the inherent inductive biases present in the convolutional architectures, they lack understanding of long-range dependencies in the image. Recently proposed Transformer-based architectures that leverage self-attention mechanism encode long-range dependencies and learn representations that are highly expressive. This motivates us to explore Transformer-based solutions and study the feasibility of using Transformer-based network architectures for medical image segmentation tasks. Majority of existing Transformer-based network architectures proposed for vision applications require large-scale datasets to train properly. However, compared to the datasets for vision applications, for medical imaging the number of data samples is relatively low, making it difficult to efficiently train transformers for medical applications. To this end, we propose a Gated Axial-Attention model which extends the existing architectures by introducing an additional control mechanism in the self-attention module. Furthermore, to train the model effectively on medical images, we propose a Local-Global training strategy (LoGo) which further improves the performance. Specifically, we operate on the whole image and patches to learn global and local features, respectively. The proposed Medical Transformer (MedT) is evaluated on three different medical image segmentation datasets and it is shown that it achieves better performance than the convolutional and other related transformer-based architectures. Code: https://github.com/jeya-maria-jose/Medical-Transformer

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Medical Image Segmentation Brain US MedT F1 88.84 # 1
IoU 81.34 # 1
Medical Image Segmentation Brain US U-Net F1 87.92 # 3
IoU 80.14 # 3
Medical Image Segmentation Brain US LoGo F1 88.54 # 2
IoU 80.84 # 2
Medical Image Segmentation GlaS U-Net F1 76.26 # 3
IoU 63.03 # 6
Medical Image Segmentation GlaS LoGo F1 79.68 # 2
IoU 67.69 # 5
Medical Image Segmentation GlaS MedT F1 81.02 # 1
IoU 69.61 # 4
Medical Image Segmentation MoNuSeg MedT F1 79.55 # 6
IoU 66.17 # 5
Medical Image Segmentation MoNuSeg U-Net F1 76.83 # 9
IoU 62.49 # 10
Medical Image Segmentation MoNuSeg LoGo F1 79.56 # 5
IoU 66.17 # 5

Methods