Panoptic Segmentation
213 papers with code • 24 benchmarks • 32 datasets
Panoptic Segmentation is a computer vision task that combines semantic segmentation and instance segmentation to provide a comprehensive understanding of the scene. The goal of panoptic segmentation is to segment the image into semantically meaningful parts or regions, while also detecting and distinguishing individual instances of objects within those regions. In a given image, every pixel is assigned a semantic label, and pixels belonging to "things" classes (countable objects with instances, like cars and people) are assigned unique instance IDs. ( Image credit: Detectron2 )
Benchmarks
These leaderboards are used to track progress in Panoptic Segmentation
| Trend | Dataset | Best Model | Paper | Code | Compare |
|---|---|---|---|---|---|
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
Libraries
Use these libraries to find Panoptic Segmentation models and implementations
MS COCO
Cityscapes
KITTI
ScanNet
ADE20K
NYUv2
SemanticKITTI
SUN RGB-D
S3DIS
BDD100K
Latest papers
DVIS++: Improved Decoupled Framework for Universal Video Segmentation
We present the \textbf{D}ecoupled \textbf{VI}deo \textbf{S}egmentation (DVIS) framework, a novel approach for the challenging task of universal video segmentation, including video instance segmentation (VIS), video semantic segmentation (VSS), and video panoptic segmentation (VPS).
MaskConver: Revisiting Pure Convolution Model for Panoptic Segmentation
With ResNet50 backbone, our MaskConver achieves 53. 6% PQ on the COCO panoptic val set, outperforming the modern convolution-based model, Panoptic FCN, by 9. 3% as well as transformer-based models such as Mask2Former (+1. 7% PQ) and kMaX-DeepLab (+0. 6% PQ).
GIVT: Generative Infinite-Vocabulary Transformers
google-research/big_vision
•
•
We introduce generative infinite-vocabulary transformers (GIVT) which generate vector sequences with real-valued entries, instead of discrete tokens from a finite vocabulary.
Aligning and Prompting Everything All at Once for Universal Visual Perception
shenyunhang/ape
•
•
However, predominant paradigms, driven by casting instance-level tasks as an object-word alignment, bring heavy cross-modality interaction, which is not effective in prompting object detection and visual grounding.
MaXTron: Mask Transformer with Trajectory Attention for Video Panoptic Segmentation
tacju/maxtron
•
•
To alleviate the issue, we propose to adapt the trajectory attention for both the dense pixel features and object queries, aiming to improve the short-term and long-term tracking results, respectively.
Panoptic Video Scene Graph Generation
jingkang50/openpvsg
•
•
CVPR 2023
PVSG relates to the existing video scene graph generation (VidSGG) problem, which focuses on temporal interactions between humans and objects grounded with bounding boxes in videos.
OneFormer3D: One Transformer for Unified Point Cloud Segmentation
Semantic, instance, and panoptic segmentation of 3D point clouds have been addressed using task-specific models of distinct design.
Center Focusing Network for Real-Time LiDAR Panoptic Segmentation
gangzhang842/cfnet
•
•
CVPR 2023
LiDAR panoptic segmentation facilitates an autonomous vehicle to comprehensively understand the surrounding objects and scenes and is required to run in real time.
Lidar Panoptic Segmentation and Tracking without Bells and Whistles
Our network is modular by design and optimized for all aspects of both the panoptic segmentation and tracking task.
Hierarchical Mask2Former: Panoptic Segmentation of Crops, Weeds and Leaves
madeleinedarbyshire/hierarchicalmask2former
•
•
We achieve a PQ{\dag} of 75. 99.
