Search Results for author:
Found 10 papers, 3 papers with code
UniCode: Learning a Unified Codebook for Multimodal Large Language Models
In this paper, we propose \textbf{UniCode}, a novel approach within the domain of multimodal large language models (MLLMs) that learns a unified codebook to efficiently tokenize visual, text, and potentially other types of signals.
POV: Prompt-Oriented View-Agnostic Learning for Egocentric Hand-Object Interaction in the Multi-View World
We humans are good at translating third-person observations of hand-object interactions (HOI) into an egocentric view.
SPAFormer: Sequential 3D Part Assembly with Transformers
We introduce SPAFormer, an innovative model designed to overcome the combinatorial explosion challenge in the 3D Part Assembly (3D-PA) task.
Steve-Eye: Equipping LLM-based Embodied Agents with Visual Perception in Open Worlds
Steve-Eye integrates the LLM with a visual encoder which enables it to process visual-text inputs and generate multimodal feedback.
LLaMA Rider: Spurring Large Language Models to Explore the Open World
Recently, various studies have leveraged Large Language Models (LLMs) to help decision-making and planning in environments, and try to align the LLMs' knowledge with the world conditions.
No-frills Temporal Video Grounding: Multi-Scale Neighboring Attention and Zoom-in Boundary Detection
Temporal video grounding (TVG) aims to retrieve the time interval of a language query from an untrimmed video.
Accommodating Audio Modality in CLIP for Multimodal Processing
In this paper, we extend the stateof-the-art Vision-Language model CLIP to accommodate the audio modality for Vision-Language-Audio multimodal processing.
Open-Category Human-Object Interaction Pre-Training via Language Modeling Framework
Human-object interaction (HOI) has long been plagued by the conflict between limited supervised data and a vast number of possible interaction combinations in real life.
Exploring Anchor-based Detection for Ego4D Natural Language Query
In this paper we provide the technique report of Ego4D natural language query challenge in CVPR 2022.
VRDFormer: End-to-End Video Visual Relation Detection With Transformers
Most previous works adopt a multi-stage framework for video visual relation detection (VidVRD), which cannot capture long-term spatiotemporal contexts in different stages and also suffers from inefficiency.
