Search Results for author:
Found 14 papers, 8 papers with code
MiniGPT4-Video: Advancing Multimodal LLMs for Video Understanding with Interleaved Visual-Textual Tokens
This paper introduces MiniGPT4-Video, a multimodal Large Language Model (LLM) designed specifically for video understanding.
MiniGPT-v2: large language model as a unified interface for vision-language multi-task learning
Motivated by this, we target to build a unified interface for completing many vision-language tasks including image description, visual question answering, and visual grounding, among others.
Ranked #10 on
Visual Question Answering
on BenchLMM
Exploring Open-Vocabulary Semantic Segmentation without Human Labels
Although acquired extensive knowledge of visual concepts, it is non-trivial to exploit knowledge from these VL models to the task of semantic segmentation, as they are usually trained at an image level.
MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models
Our work, for the first time, uncovers that properly aligning the visual features with an advanced large language model can possess numerous advanced multi-modal abilities demonstrated by GPT-4, such as detailed image description generation and website creation from hand-drawn drafts.
Ranked #9 on
Visual Question Answering
on BenchLMM
Video ChatCaptioner: Towards Enriched Spatiotemporal Descriptions
Video captioning aims to convey dynamic scenes from videos using natural language, facilitating the understanding of spatiotemporal information within our environment.
ChatGPT Asks, BLIP-2 Answers: Automatic Questioning Towards Enriched Visual Descriptions
By keeping acquiring new visual information from BLIP-2's answers, ChatCaptioner is able to generate more enriched image descriptions.
Guiding Online Reinforcement Learning with Action-Free Offline Pretraining
In this paper, we investigate the potential of using action-free offline datasets to improve online reinforcement learning, name this problem Reinforcement Learning with Action-Free Offline Pretraining (AFP-RL).
Exploring Open-Vocabulary Semantic Segmentation from CLIP Vision Encoder Distillation Only
Semantic segmentation is a crucial task in computer vision that involves segmenting images into semantically meaningful regions at the pixel level.
Value Memory Graph: A Graph-Structured World Model for Offline Reinforcement Learning
In some complex environments with continuous state-action spaces, sparse rewards, and/or long temporal horizons, learning a good policy in the original environments can be difficult.
Social-Implicit: Rethinking Trajectory Prediction Evaluation and The Effectiveness of Implicit Maximum Likelihood Estimation
AMD is a metric that quantifies how close the whole generated samples are to the ground truth.
Ranked #1 on
Trajectory Prediction
on Stanford Drone
(ADE (in world coordinates) metric)
CausalDyna: Improving Generalization of Dyna-style Reinforcement Learning via Counterfactual-Based Data Augmentation
Deep reinforcement learning agents trained in real-world environments with a limited diversity of object properties to learn manipulation tasks tend to suffer overfitting and fail to generalize to unseen testing environments.
RelTransformer: A Transformer-Based Long-Tail Visual Relationship Recognition
This paper shows that modeling an effective message-passing flow through an attention mechanism can be critical to tackling the compositionality and long-tail challenges in VRR.
HalentNet: Multimodal Trajectory Forecasting with Hallucinative Intents
Our model's learned representation leads to better and more semantically meaningful coverage of the trajectory distribution.
Motion Forecasting with Unlikelihood Training
We propose a new objective, unlikelihood training, which forces generated trajectories that conflicts with contextual information to be assigned a lower probability by our model.
