Search Results for author:
Found 16 papers, 7 papers with code
3D Open-Vocabulary Panoptic Segmentation with 2D-3D Vision-Language Distillation
3D panoptic segmentation is a challenging perception task, especially in autonomous driving.
Detection-Oriented Image-Text Pretraining for Open-Vocabulary Detection
We present a new open-vocabulary detection approach based on detection-oriented image-text pretraining to bridge the gap between image-level pretraining and open-vocabulary object detection.
Ranked #1 on
Open Vocabulary Object Detection
on LVIS v1.0
Contrastive Feature Masking Open-Vocabulary Vision Transformer
We present Contrastive Feature Masking Vision Transformer (CFM-ViT) - an image-text pretraining methodology that achieves simultaneous learning of image- and region-level representation for open-vocabulary object detection (OVD).
Ranked #5 on
Open Vocabulary Object Detection
on LVIS v1.0
Region-Aware Pretraining for Open-Vocabulary Object Detection with Vision Transformers
We present Region-aware Open-vocabulary Vision Transformers (RO-ViT) - a contrastive image-text pretraining recipe to bridge the gap between image-level pretraining and open-vocabulary object detection.
Ranked #5 on
Zero-Shot Cross-Modal Retrieval
on Flickr30k
RECLIP: Resource-efficient CLIP by Training with Small Images
We present RECLIP (Resource-efficient CLIP), a simple method that minimizes computational resource footprint for CLIP (Contrastive Language Image Pretraining).
MaMMUT: A Simple Architecture for Joint Learning for MultiModal Tasks
We propose a novel paradigm of training with a decoder-only model for multimodal tasks, which is surprisingly effective in jointly learning of these disparate vision-language tasks.
Ranked #1 on
Video Captioning
on MSVD
Rethinking Video ViTs: Sparse Video Tubes for Joint Image and Video Learning
We present a simple approach which can turn a ViT encoder into an efficient video model, which can seamlessly work with both image and video inputs.
Ranked #2 on
Action Classification
on Kinetics-600
(using extra training data)
F-VLM: Open-Vocabulary Object Detection upon Frozen Vision and Language Models
We present F-VLM, a simple open-vocabulary object detection method built upon Frozen Vision and Language Models.
PaLI: A Jointly-Scaled Multilingual Language-Image Model
PaLI generates text based on visual and textual inputs, and with this interface performs many vision, language, and multimodal tasks, in many languages.
Pre-training image-language transformers for open-vocabulary tasks
We present a pre-training approach for vision and language transformer models, which is based on a mixture of diverse tasks.
Video Question Answering with Iterative Video-Text Co-Tokenization
Video question answering is a challenging task that requires understanding jointly the language input, the visual information in individual video frames, as well as the temporal information about the events occurring in the video.
Ranked #4 on
Video Question Answering
on iVQA
Answer-Me: Multi-Task Open-Vocabulary Visual Question Answering
We present Answer-Me, a task-aware multi-task framework which unifies a variety of question answering tasks, such as, visual question answering, visual entailment, visual reasoning.
FindIt: Generalized Localization with Natural Language Queries
We propose FindIt, a simple and versatile framework that unifies a variety of visual grounding and localization tasks including referring expression comprehension, text-based localization, and object detection.
Patch2CAD: Patchwise Embedding Learning for In-the-Wild Shape Retrieval from a Single Image
3D perception of object shapes from RGB image input is fundamental towards semantic scene understanding, grounding image-based perception in our spatially 3-dimensional real-world environments.
Learning Open-World Object Proposals without Learning to Classify
In this paper, we identify that the problem is that the binary classifiers in existing proposal methods tend to overfit to the training categories.
Ranked #2 on
Open World Object Detection
on COCO VOC to non-VOC
Open-vocabulary Object Detection via Vision and Language Knowledge Distillation
On COCO, ViLD outperforms the previous state-of-the-art by 4. 8 on novel AP and 11. 4 on overall AP.
Ranked #2 on
Open Vocabulary Object Detection
on Objects365
