One-Shot Video Object Segmentation

CVPR 2017 β€’ Sergi Caelles β€’ Kevis-Kokitsi Maninis β€’ Jordi Pont-Tuset β€’ Laura Leal-TaixΓ© β€’ Daniel Cremers β€’ Luc Van Gool

This paper tackles the task of semi-supervised video object segmentation, i.e., the separation of an object from the background in a video, given the mask of the first frame. We present One-Shot Video Object Segmentation (OSVOS), based on a fully-convolutional neural network architecture that is able to successively transfer generic semantic information, learned on ImageNet, to the task of foreground segmentation, and finally to learning the appearance of a single annotated object of the test sequence (hence one-shot)... (read more)

PDF Abstract CVPR 2017 PDF CVPR 2017 Abstract

Results from the Paper


TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK RESULT BENCHMARK
Semi-Supervised Video Object Segmentation DAVIS 2016 OSVOS Jaccard (Mean) 79.8 # 18
Jaccard (Recall) 93.6 # 13
Jaccard (Decay) 14.9 # 4
F-measure (Mean) 80.6 # 16
F-measure (Recall) 92.6 # 8
F-measure (Decay) 15.0 # 23
J&F 80.2 # 18
Semi-Supervised Video Object Segmentation DAVIS 2017 (test-dev) OSVOS J&F 50.9 # 15
Jaccard (Mean) 47.0 # 17
Jaccard (Recall) 52.1 # 16
Jaccard (Decay) 19.2 # 5
F-measure (Mean) 54.8 # 15
F-measure (Recall) 59.7 # 16
F-measure (Decay) 19.8 # 6
Semi-Supervised Video Object Segmentation DAVIS 2017 (val) OSVOS Jaccard (Mean) 56.6 # 18
Jaccard (Recall) 63.8 # 16
Jaccard (Decay) 26.1 # 18
F-measure (Mean) 63.9 # 16
F-measure (Recall) 73.8 # 14
F-measure (Decay) 27.0 # 18
J&F 60.25 # 19
Semi-Supervised Video Object Segmentation YouTube-VOS OSVOS F-Measure (Seen) 60.5 # 5
F-Measure (Unseen) 60.7 # 3
Overall 58.8 # 4
Speed (FPS) 0.10 # 4
Jaccard (Seen) 59.8 # 6
Jaccard (Unseen) 54.2 # 3
Visual Object Tracking YouTube-VOS OSVOS O (Average of Measures) 58.8 # 1
F-Measure (Seen) 60.5 # 2
F-Measure (Unseen) 60.7 # 1
Youtube-VOS YouTube-VOS OSVOS F-Measure (Seen) 60.5 # 2
F-Measure (Unseen) 60.7 # 1
Jaccard (Seen) 59.8 # 3
Jaccard (Unseen) 54.2 # 1

Results from Other Papers


TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK SOURCE PAPER COMPARE
Semi-Supervised Video Object Segmentation YouTube OSVOS mIoU 0.783 # 4

Methods used in the Paper


METHOD TYPE
πŸ€– No Methods Found Help the community by adding them if they're not listed; e.g. Deep Residual Learning for Image Recognition uses ResNet