Search Results for author:
Found 13 papers, 9 papers with code
MagicTime: Time-lapse Video Generation Models as Metamorphic Simulators
Recent advances in Text-to-Video generation (T2V) have achieved remarkable success in synthesizing high-quality general videos from textual descriptions.
LLMBind: A Unified Modality-Task Integration Framework
In the multi-modal domain, the dependence of various models on specific input formats leads to user confusion and hinders progress.
MoE-LLaVA: Mixture of Experts for Large Vision-Language Models
In this work, we propose a simple yet effective training strategy MoE-Tuning for LVLMs.
Ranked #57 on
Visual Question Answering
on MM-Vet
Continuous-Multiple Image Outpainting in One-Step via Positional Query and A Diffusion-based Approach
At inference, we generate images with arbitrary expansion multiples by inputting an anchor image and its corresponding positional embeddings.
GPT-4V(ision) as A Social Media Analysis Engine
Our investigation begins with a preliminary quantitative analysis for each task using existing benchmark datasets, followed by a careful review of the results and a selection of qualitative samples that illustrate GPT-4V's potential in understanding multimodal social media content.
A Survey of Large Language Models in Medicine: Progress, Application, and Challenge
Therefore, this review aims to provide a detailed overview of the development and deployment of LLMs in medicine, including the challenges and opportunities they face.
Improving Scene Graph Generation with Superpixel-Based Interaction Learning
(ii) We explore intra-entity and cross-entity interactions among the superpixels to enrich fine-grained interactions between entities at an earlier stage.
Text-Video Retrieval with Disentangled Conceptualization and Set-to-Set Alignment
In this paper, we propose the Disentangled Conceptualization and Set-to-set Alignment (DiCoSA) to simulate the conceptualizing and reasoning process of human beings.
Cross-Modality Time-Variant Relation Learning for Generating Dynamic Scene Graphs
In this paper, we propose a Time-variant Relation-aware TRansformer (TR$^2$), which aims to model the temporal change of relations in dynamic scene graphs.
Video-Text as Game Players: Hierarchical Banzhaf Interaction for Cross-Modal Representation Learning
Contrastive learning-based video-language representation learning approaches, e. g., CLIP, have achieved outstanding performance, which pursue semantic interaction upon pre-defined video-text pairs.
Ranked #8 on
Video Question Answering
on MSRVTT-QA
Expectation-Maximization Contrastive Learning for Compact Video-and-Language Representations
Most video-and-language representation learning approaches employ contrastive learning, e. g., CLIP, to project the video and text features into a common latent space according to the semantic similarities of text-video pairs.
Ranked #2 on
Video Retrieval
on LSMDC
(text-to-video Mean Rank metric)
Toward 3D Spatial Reasoning for Human-like Text-based Visual Question Answering
Under this setting, these 2D spatial reasoning approaches cannot distinguish the fine-grain spatial relations between visual objects and scene texts on the same image plane, thereby impairing the interpretability and performance of TextVQA models.
Guoym at SemEval-2020 Task 8: Ensemble-based Classification of Visuo-Lingual Metaphor in Memes
In our system, we utilize five types of representation of data as input of base classifiers to extract information from different aspects.
