Search Results for author:
Found 23 papers, 15 papers with code
Delocate: Detection and Localization for Deepfake Videos with Randomly-Located Tampered Traces
Deepfake videos are becoming increasingly realistic, showing subtle tampering traces on facial areasthat vary between frames.
ASSISTGUI: Task-Oriented Desktop Graphical User Interface Automation
Graphical User Interface (GUI) automation holds significant promise for assisting users with complex tasks, thereby boosting human productivity.
ViT-Lens: Towards Omni-modal Representations
In this paper, we present ViT-Lens-2 that facilitates efficient omni-modal representation learning by perceiving novel modalities with a pretrained ViT and aligning them to a pre-defined space.
CVPR 2023 Text Guided Video Editing Competition
In this paper we present a retrospective on the competition and describe the winning method.
Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generation
In this paper, we are the first to propose a hybrid model, dubbed as Show-1, which marries pixel-based and latent-based VDMs for text-to-video generation.
Ranked #2 on
Text-to-Video Generation
on EvalCrafter Text-to-Video (ECTV) Dataset
(using extra training data)
Recap: Detecting Deepfake Video with Unpredictable Tampered Traces via Recovering Faces and Mapping Recovered Faces
In the recovering stage, the model focuses on randomly masking regions of interest (ROIs) and reconstructing real faces without unpredictable tampered traces, resulting in a relatively good recovery effect for real faces while a poor recovery effect for fake faces.
UniVTG: Towards Unified Video-Language Temporal Grounding
Most methods in this direction develop taskspecific models that are trained with type-specific labels, such as moment retrieval (time interval) and highlight detection (worthiness curve), which limits their abilities to generalize to various VTG tasks and labels.
Ranked #3 on
Natural Language Moment Retrieval
on TACoS
GroundNLQ @ Ego4D Natural Language Queries Challenge 2023
Motivated by this, we leverage a two-stage pre-training strategy to train egocentric feature extractors and the grounding model on video narrations, and further fine-tune the model on annotated data.
AssistGPT: A General Multi-modal Assistant that can Plan, Execute, Inspect, and Learn
2) Flexible inputs and intermediate results.
Affordance Grounding from Demonstration Video to Target Image
Humans excel at learning from expert demonstrations and solving their own problems.
Ranked #1 on
Video-to-image Affordance Grounding
on EPIC-Hotspot
Mover: Mask and Recovery based Facial Part Consistency Aware Method for Deepfake Video Detection
Specifically, given a real face image, we first pretrain a masked autoencoder to learn facial part consistency by dividing faces into three parts and randomly masking ROIs, which are then recovered based on the unmasked facial parts.
MIST: Multi-modal Iterative Spatial-Temporal Transformer for Long-form Video Question Answering
To build Video Question Answering (VideoQA) systems capable of assisting humans in daily activities, seeking answers from long-form videos with diverse and complex events is a must.
Ranked #2 on
Video Question Answering
on AGQA 2.0 balanced
An Efficient COarse-to-fiNE Alignment Framework @ Ego4D Natural Language Queries Challenge 2022
This technical report describes the CONE approach for Ego4D Natural Language Queries (NLQ) Challenge in ECCV 2022.
CONE: An Efficient COarse-to-fiNE Alignment Framework for Long Video Temporal Grounding
This paper tackles an emerging and challenging problem of long video temporal grounding~(VTG) that localizes video moments related to a natural language (NL) query.
Symbolic Replay: Scene Graph as Prompt for Continual Learning on VQA Task
However, CL on VQA involves not only the expansion of label sets (new Answer sets).
Egocentric Video-Language Pretraining @ Ego4D Challenge 2022
In this report, we propose a video-language pretraining (VLP) based solution \cite{kevin2022egovlp} for four Ego4D challenge tasks, including Natural Language Query (NLQ), Moment Query (MQ), Object State Change Classification (OSCC), and PNR Localization (PNR).
Egocentric Video-Language Pretraining
Video-Language Pretraining (VLP), which aims to learn transferable representation to advance a wide range of video-text downstream tasks, has recently received increasing attention.
GEB+: A Benchmark for Generic Event Boundary Captioning, Grounding and Retrieval
In this paper, we introduce a new dataset called Kinetic-GEB+.
Ranked #1 on
Boundary Captioning
on Kinetics-GEB+
AssistQ: Affordance-centric Question-driven Task Completion for Egocentric Assistant
In this paper, we define a new task called Affordance-centric Question-driven Task Completion, where the AI assistant should learn from instructional videos to provide step-by-step help in the user's view.
AssistSR: Task-oriented Video Segment Retrieval for Personal AI Assistant
In contrast, we present a new task called Task-oriented Question-driven Video Segment Retrieval (TQVSR).
Env-QA: A Video Question Answering Benchmark for Comprehensive Understanding of Dynamic Environments
Visual understanding goes well beyond the study of images or videos on the web.
Multi-Modal Graph Neural Network for Joint Reasoning on Vision and Scene Text
Then, we introduce three aggregators which guide the message passing from one graph to another to utilize the contexts in various modalities, so as to refine the features of nodes.
CRIC: A VQA Dataset for Compositional Reasoning on Vision and Commonsense
To comprehensively evaluate such abilities, we propose a VQA benchmark, CRIC, which introduces new types of questions about Compositional Reasoning on vIsion and Commonsense, and an evaluation metric integrating the correctness of answering and commonsense grounding.
