Search Results for author:
Found 41 papers, 16 papers with code
SimAug: Learning Robust Representations from Simulation for Trajectory Prediction
We approach this problem through the real-data-free setting in which the model is trained only on 3D simulation data and applied out-of-the-box to a wide variety of real cameras.
Ranked #1 on
Trajectory Forecasting
on ActEV
Direct Preference Optimization of Video Large Multimodal Models from Language Model Reward
Preference modeling techniques, such as direct preference optimization (DPO), has shown effective in enhancing the generalization abilities of large language model (LLM).
Towards Calibrated Robust Fine-Tuning of Vision-Language Models
Robust fine-tuning aims to ensure performance on out-of-distribution (OOD) samples, which is sometimes compromised by pursuing adaptation on in-distribution (ID) samples.
Hyperbolic vs Euclidean Embeddings in Few-Shot Learning: Two Sides of the Same Coin
Recent research in representation learning has shown that hierarchical data lends itself to low-dimensional and highly informative representations in hyperbolic space.
STMT: A Spatial-Temporal Mesh Transformer for MoCap-Based Action Recognition
The model uses a hierarchical transformer with intra-frame off-set attention and inter-frame self-attention.
Multilingual Multimodal Pre-training for Zero-Shot Cross-Lingual Transfer of Vision-Language Models
Specifically, we focus on multilingual text-to-video search and propose a Transformer-based model that learns contextualized multilingual multimodal embeddings.
Learning To Hallucinate Examples From Extrinsic and Intrinsic Supervision
Learning to hallucinate additional examples has recently been shown as a promising direction to address few-shot learning tasks.
Spatial-Temporal Alignment Network for Action Recognition and Detection
The experimental results show that the STAN model can consistently improve the state of the arts in both action detection and action recognition tasks.
Event-Related Bias Removal for Real-time Disaster Events
Furthermore, the classification of information in real-time systems requires training on out-of-domain data, as we do not have any data from a new emerging crisis.
Support-set bottlenecks for video-text representation learning
The dominant paradigm for learning video-text representations -- noise contrastive learning -- increases the similarity of the representations of pairs of samples that are known to be related, such as text and video from the same sample, and pushes away the representations of all other pairs.
Robust Long-Term Object Tracking via Improved Discriminative Model Prediction
We propose an improved discriminative model prediction method for robust long-term tracking based on a pre-trained short-term tracker.
From A Glance to "Gotcha": Interactive Facial Image Retrieval with Progressive Relevance Feedback
Facial image retrieval plays a significant role in forensic investigations where an untrained witness tries to identify a suspect from a massive pool of images.
MSNet: A Multilevel Instance Segmentation Network for Natural Disaster Damage Assessment in Aerial Videos
This provides the first benchmark for quantitative evaluation of models to assess building damage using aerial videos.
Unsupervised Multimodal Neural Machine Translation with Pseudo Visual Pivoting
In this paper, we investigate how to utilize visual content for disambiguation and promoting latent space alignment in unsupervised MMT.
SimAug: Learning Robust Representations from 3D Simulation for Pedestrian Trajectory Prediction in Unseen Cameras
We refer to our method as SimAug.
Ranked #2 on
Trajectory Prediction
on ActEV
ZSTAD: Zero-Shot Temporal Activity Detection
An integral part of video analysis and surveillance is temporal activity detection, which means to simultaneously recognize and localize activities in long untrimmed videos.
Gun Source and Muzzle Head Detection
We have interesting results both in bounding the shooter as well as detecting the gun smoke.
The Garden of Forking Paths: Towards Multi-Future Trajectory Prediction
The first contribution is a new dataset, created in a realistic 3D simulator, which is based on real world trajectory data, and then extrapolated by human annotators to achieve different latent goals.
Ranked #1 on
Multi-future Trajectory Prediction
on ForkingPaths
Multi-Head Attention with Diversity for Learning Grounded Multilingual Multimodal Representations
With the aim of promoting and understanding the multilingual version of image search, we leverage visual object detection and propose a model with diverse multi-head attention to learn grounded multilingual multimodal representations.
Improving the Learning of Multi-column Convolutional Neural Network for Crowd Counting
By minimizing the mutual information, each column is guided to learn features with different image scales.
Learning Spatial Awareness to Improve Crowd Counting
Although the Maximum Excess over SubArrays (MESA) loss has been previously proposed to address the above issues by finding the rectangular subregion whose predicted density map has the maximum difference from the ground truth, it cannot be solved by gradient descent, thus can hardly be integrated into the deep learning framework.
Ranked #5 on
Crowd Counting
on WorldExpo’10
Activitynet 2019 Task 3: Exploring Contexts for Dense Captioning Events in Videos
The overall system achieves the state-of-the-art performance on the dense-captioning events in video task with 9. 91 METEOR score on the challenge testing set.
Unsupervised Bilingual Lexicon Induction from Mono-lingual Multimodal Data
The linguistic feature is learned from the sentence contexts with visual semantic constraints, which is beneficial to learn translation for words that are less visual-relevant.
Technical Report of the Video Event Reconstruction and Analysis (VERA) System -- Shooter Localization, Models, Interface, and Beyond
Among other uses, VERA enables the localization of a shooter from just a few videos that include the sound of gunshots.
ExCL: Extractive Clip Localization Using Natural Language Descriptions
The task of retrieving clips within videos based on a given natural language query requires cross-modal reasoning over multiple frames.
Peeking into the Future: Predicting Future Person Activities and Locations in Videos
To facilitate the training, the network is learned with an auxiliary task of predicting future location in which the activity will happen.
Ranked #1 on
Activity Prediction
on ActEV
Traffic Danger Recognition With Surveillance Cameras Without Training Data
Therefore, we developed a model to predict and identify car crashes from surveillance cameras based on a 3D reconstruction of the road plane and prediction of trajectories.
Perceiving Physical Equation by Observing Visual Scenarios
To tackle this challenge, we present a novel pipeline comprised of an Observer Engine and a Physicist Engine by respectively imitating the actions of an observer and a physicist in the real world.
CADP: A Novel Dataset for CCTV Traffic Camera based Accident Analysis
Our experiments indicate a considerable improvement in object detection accuracy: +8. 51% for CM and +6. 20% for ACM.
Activity Recognition on a Large Scale in Short Videos - Moments in Time Dataset
Moments capture a huge part of our lives.
Stacked Pooling: Improving Crowd Counting by Boosting Scale Invariance
In this work, we explore the cross-scale similarity in crowd counting scenario, in which the regions of different scales often exhibit high visual similarity.
RUC+CMU: System Report for Dense Captioning Events in Videos
This notebook paper presents our system in the ActivityNet Dense Captioning in Video task (task 3).
Focal Visual-Text Attention for Visual Question Answering
Recent insights on language and vision with neural networks have been successfully applied to simple single-image visual question answering.
Ranked #1 on
Memex Question Answering
on MemexQA
GNAS: A Greedy Neural Architecture Search Method for Multi-Attribute Learning
A key problem in deep multi-attribute learning is to effectively discover the inter-attribute correlation structures.
Video Captioning with Guidance of Multimodal Latent Topics
For the topic prediction task, we use the mined topics as the teacher to train a student topic prediction model, which learns to predict the latent topics from multimodal contents of videos.
MemexQA: Visual Memex Question Answering
This paper proposes a new task, MemexQA: given a collection of photos or videos from a user, the goal is to automatically answer questions that help users recover their memory about events captured in the collection.
Exploiting Multi-modal Curriculum in Noisy Web Data for Large-scale Concept Learning
Learning video concept detectors automatically from the big but noisy web data with no additional manual annotations is a novel but challenging area in the multimedia and the machine learning community.
The Solution Path Algorithm for Identity-Aware Multi-Object Tracking
The tracker is formulated as a quadratic optimization problem with L0 norm constraints, which we propose to solve with the solution path algorithm.
The Best of Both Worlds: Combining Data-independent and Data-driven Approaches for Action Recognition
First, we propose a two-stream Stacked Convolutional Independent Subspace Analysis (ConvISA) architecture to show that unsupervised learning methods can significantly boost the performance of traditional local features extracted from data-independent models.
Self-Paced Learning with Diversity
Self-paced learning (SPL) is a recently proposed learning regime inspired by the learning process of humans and animals that gradually incorporates easy to more complex samples into training.
Harry Potter's Marauder's Map: Localizing and Tracking Multiple Persons-of-Interest by Nonnegative Discretization
A device just like Harry Potter's Marauder's Map, which pinpoints the location of each person-of-interest at all times, provides invaluable information for analysis of surveillance videos.
