Auto-Prompting SAM for Mobile Friendly 3D Medical Image Segmentation

Segment Anything Model (SAM) has rapidly been adopted for segmenting a wide range of natural images. However, recent studies have indicated that SAM exhibits subpar performance on 3D medical image segmentation tasks. In addition to the domain gaps between natural and medical images, disparities in the spatial arrangement between 2D and 3D images, the substantial computational burden imposed by powerful GPU servers, and the time-consuming manual prompt generation impede the extension of SAM to a broader spectrum of medical image segmentation applications. To mitigate these challenges, we introduce a novel method, AutoSAM Adapter, designed specifically for 3D multi-organ CT-based segmentation. This approach utilizes parameter-efficient adaptation techniques and an automatic prompt learning paradigm, transforming SAM's capabilities for 3D medical image segmentation. It eliminates the need for manual prompts and achieves SOTA performance in CT-based multi-organ segmentation tasks. Furthermore, we successfully transfer the acquired knowledge of the AutoSAM Adapter to other lightweight models tailored for 3D medical image analysis with enhanced performance. Through extensive experiments, the AutoSAM Adapter has been demonstrated as an effective method to adapt the foundational SAM-based 2D natural image segmentation model for 3D medical image segmentation tasks.