MST: Adaptive Multi-Scale Tokens Guided Interactive Segmentation
Interactive segmentation has gained significant attention for its application in human-computer interaction and data annotation. To address the target scale variation issue in interactive segmentation, a novel multi-scale token adaptation algorithm is proposed. By performing top-k operations across multi-scale tokens, the computational complexity is greatly simplified while ensuring performance. To enhance the robustness of multi-scale token selection, we also propose a token learning algorithm based on contrastive loss. This algorithm can effectively improve the performance of multi-scale token adaptation. Extensive benchmarking shows that the algorithm achieves state-of-the-art (SOTA) performance, compared to current methods. An interactive demo and all reproducible codes will be released at https://github.com/hahamyt/mst.
PDF Abstract
MS COCO
DAVIS
LVIS
SBD
LoveDA
| Task | Dataset | Model | Metric Name | Metric Value | Global Rank | Benchmark |
|---|---|---|---|---|---|---|
| Interactive Segmentation | Berkeley | ViT-B+MST+CL | NoC@90 | 1.50 | # 3 | |
| Interactive Segmentation | COCO minival | ViT-B+MST+CL | NoC@85 | 2.08 | # 1 | |
| NoC@90 | 2.85 | # 1 | ||||
| Interactive Segmentation | DAVIS | ViT-B+MST+CL | NoC@90 | 4.55 | # 3 | |
| Interactive Segmentation | DAVIS-585 | ViT-B+MST+CL | NoC@90 | 2.29 | # 1 | |
| NoC@85 | 1.80 | # 1 | ||||
| Interactive Segmentation | GrabCut | ViT-B+MST+CL | NoC@90 | 1.48 | # 6 | |
| Interactive Segmentation | PascalVOC | ViT-B+MST+CL | NoC@85 | 1.69 | # 1 | |
| NoC@90 | 1.90 | # 1 | ||||
| Interactive Segmentation | SBD | ViT-B+MST+CL | NoC@85 | 3.03 | # 4 |
