Search Results for author:
Found 23 papers, 7 papers with code
Distilling Vision-Language Models on Millions of Videos
Our best model outperforms state-of-the-art methods on MSR-VTT zero-shot text-to-video retrieval by 6%.
GeomVerse: A Systematic Evaluation of Large Models for Geometric Reasoning
In this paper, we evaluate the reasoning capabilities of VLMs along various axes through the lens of geometry problems.
Omni-SMoLA: Boosting Generalist Multimodal Models with Soft Mixture of Low-rank Experts
Large multi-modal models (LMMs) exhibit remarkable performance across numerous tasks.
Ranked #1 on
Visual Question Answering (VQA)
on A-OKVQA
(using extra training data)
Non-Intrusive Adaptation: Input-Centric Parameter-efficient Fine-Tuning for Versatile Multimodal Modeling
Intrusive PEFT techniques directly change a model's internal architecture.
PaLI-3 Vision Language Models: Smaller, Faster, Stronger
This paper presents PaLI-3, a smaller, faster, and stronger vision language model (VLM) that compares favorably to similar models that are 10x larger.
Ranked #2 on
Temporal/Casual QA
on NExT-QA
(using extra training data)
CausalLM is not optimal for in-context learning
Recent empirical evidence indicates that transformer based in-context learning performs better when using a prefix language model (prefixLM), in which in-context samples can all attend to each other, compared to causal language models (causalLM), which use auto-regressive attention that prohibits in-context samples to attend to future samples.
RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control
Our goal is to enable a single end-to-end trained model to both learn to map robot observations to actions and enjoy the benefits of large-scale pretraining on language and vision-language data from the web.
PaLI-X: On Scaling up a Multilingual Vision and Language Model
We present the training recipe and results of scaling up PaLI-X, a multilingual vision and language model, both in terms of size of the components and the breadth of its training task mixture.
Ranked #1 on
Fine-Grained Image Recognition
on OVEN
Entity-Focused Dense Passage Retrieval for Outside-Knowledge Visual Question Answering
To address these issues, we propose an Entity-Focused Retrieval (EnFoRe) model that provides stronger supervision during training and recognizes question-relevant entities to help retrieve more specific knowledge.
Possibilities and Implications of the Multi-AI Competition
The possibility of super-AIs taking over the world has been intensively studied by numerous scholars.
Breaking Down Questions for Outside-Knowledge VQA
While general Visual Question Answering (VQA) focuses on querying visual content within an image, there is a recent trend towards Knowledge-Based VQA (KB-VQA) where a system needs to link some aspects of the question to different types of knowledge beyond the image, such as commonsense concepts and factual information.
Multi-Modal Answer Validation for Knowledge-Based VQA
Instead of searching for the answer in a vast collection of often irrelevant facts as most existing approaches do, MAVEx aims to learn how to extract relevant knowledge from noisy sources, which knowledge source to trust for each answer candidate, and how to validate the candidate using that source.
Visual Question Answering based on Local-Scene-Aware Referring Expression Generation
We quantitatively and qualitatively evaluated the proposed method on the VQA v2 dataset and compared it with state-of-the-art methods in terms of answer prediction.
CoNAN: A Complementary Neighboring-based Attention Network for Referring Expression Generation
These highly-related neighbors are determined by an attentional ranking module, as complementary features, highlighting the discriminating aspects for the target object.
Improving VQA and its Explanations \\ by Comparing Competing Explanations
Most recent state-of-the-art Visual Question Answering (VQA) systems are opaque black boxes that are only trained to fit the answer distribution given the question and visual content.
Hidden State Guidance: Improving Image Captioning using An Image Conditioned Autoencoder
Most RNN-based image captioning models receive supervision on the output words to mimic human captions.
Generating Question Relevant Captions to Aid Visual Question Answering
Visual question answering (VQA) and image captioning require a shared body of general knowledge connecting language and vision.
Ranked #31 on
Visual Question Answering (VQA)
on VQA v2 test-std
Self-Critical Reasoning for Robust Visual Question Answering
Visual Question Answering (VQA) deep-learning systems tend to capture superficial statistical correlations in the training data because of strong language priors and fail to generalize to test data with a significantly different question-answer (QA) distribution.
Ranked #6 on
Visual Question Answering (VQA)
on VQA-CP
Image Score: How to Select Useful Samples
There has long been debates on how we could interpret neural networks and understand the decisions our models make.
Faithful Multimodal Explanation for Visual Question Answering
AI systems' ability to explain their reasoning is critical to their utility and trustworthiness.
Ranked #5 on
Explanatory Visual Question Answering
on GQA-REX
Joint Image Captioning and Question Answering
Answering visual questions need acquire daily common knowledge and model the semantic connection among different parts in images, which is too difficult for VQA systems to learn from images with the only supervision from answers.
Dynamic Filtering with Large Sampling Field for ConvNets
We propose a dynamic filtering strategy with large sampling field for ConvNets (LS-DFN), where the position-specific kernels learn from not only the identical position but also multiple sampled neighbor regions.
Action Recognition with Joint Attention on Multi-Level Deep Features
We propose a novel deep supervised neural network for the task of action recognition in videos, which implicitly takes advantage of visual tracking and shares the robustness of both deep Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN).
Action Recognition In Videos
Temporal Action Localization
+1
