Open Vocabulary Object Detection
56 papers with code • 4 benchmarks • 6 datasets
Open-vocabulary detection (OVD) aims to generalize beyond the limited number of base classes labeled during the training phase. The goal is to detect novel classes defined by an unbounded (open) vocabulary at inference.
Benchmarks
These leaderboards are used to track progress in Open Vocabulary Object Detection
| Trend | Dataset | Best Model | Paper | Code | Compare |
|---|---|---|---|---|---|
|
|||||
|
|||||
|
|||||
|
Libraries
Use these libraries to find Open Vocabulary Object Detection models and implementations
MS COCO
LVIS
Objects365
MSCOCO
OVAD benchmark
Description Detection Dataset
Most implemented papers
Open-vocabulary Object Detection via Vision and Language Knowledge Distillation
On COCO, ViLD outperforms the previous state-of-the-art by 4. 8 on novel AP and 11. 4 on overall AP.
PointCLIP: Point Cloud Understanding by CLIP
zrrskywalker/pointclip
•
•
CVPR 2022
On top of that, we design an inter-view adapter to better extract the global feature and adaptively fuse the few-shot knowledge learned from 3D into CLIP pre-trained in 2D.
Simple Open-Vocabulary Object Detection with Vision Transformers
Combining simple architectures with large-scale pre-training has led to massive improvements in image classification.
Open Vocabulary Object Detection with Proposal Mining and Prediction Equalization
Open-vocabulary object detection (OVD) aims to scale up vocabulary size to detect objects of novel categories beyond the training vocabulary.
PointCLIP V2: Prompting CLIP and GPT for Powerful 3D Open-world Learning
yangyangyang127/pointclip_v2
•
•
ICCV 2023
In this paper, we first collaborate CLIP and GPT to be a unified 3D open-world learner, named as PointCLIP V2, which fully unleashes their potential for zero-shot 3D classification, segmentation, and detection.
Region-Aware Pretraining for Open-Vocabulary Object Detection with Vision Transformers
google-research/google-research
•
•
CVPR 2023
We present Region-aware Open-vocabulary Vision Transformers (RO-ViT) - a contrastive image-text pretraining recipe to bridge the gap between image-level pretraining and open-vocabulary object detection.
Described Object Detection: Liberating Object Detection with Flexible Expressions
charles-xie/awesome-described-object-detection
•
•
NeurIPS 2023
In this paper, we advance them to a more practical setting called Described Object Detection (DOD) by expanding category names to flexible language expressions for OVD and overcoming the limitation of REC only grounding the pre-existing object.
Taming Self-Training for Open-Vocabulary Object Detection
This work identifies two challenges of using self-training in OVD: noisy PLs from VLMs and frequent distribution changes of PLs.
Is CLIP the main roadblock for fine-grained open-world perception?
Modern applications increasingly demand flexible computer vision models that adapt to novel concepts not encountered during training.
Retrieval-Augmented Open-Vocabulary Object Detection
Specifically, RALF consists of two modules: Retrieval Augmented Losses (RAL) and Retrieval-Augmented visual Features (RAF).
