Search Results for author:
Found 41 papers, 22 papers with code
VideoPrism: A Foundational Visual Encoder for Video Understanding
We introduce VideoPrism, a general-purpose video encoder that tackles diverse video understanding tasks with a single frozen model.
Distilling Vision-Language Models on Millions of Videos
Our best model outperforms state-of-the-art methods on MSR-VTT zero-shot text-to-video retrieval by 6%.
Generating Enhanced Negatives for Training Language-Based Object Detectors
The recent progress in language-based open-vocabulary object detection can be largely attributed to finding better ways of leveraging large-scale data with free-form text annotations.
Unwinding Stochastic Order Flow: When to Warehouse Trades
We study how to unwind stochastic order flow with minimal transaction costs.
Deep Deformable Models: Learning 3D Shape Abstractions with Part Consistency
The task of shape abstraction with semantic part consistency is challenging due to the complex geometries of natural objects.
Learning from Semantic Alignment between Unpaired Multiviews for Egocentric Video Recognition
To facilitate the data efficiency of multiview learning, we further perform video-text alignment for first-person and third-person videos, to fully leverage the semantic knowledge to improve video representations.
Taming Self-Training for Open-Vocabulary Object Detection
This work identifies two challenges of using self-training in OVD: noisy PLs from VLMs and frequent distribution changes of PLs.
VideoGLUE: Video General Understanding Evaluation of Foundation Models
We evaluate existing foundation models video understanding capabilities using a carefully designed experiment protocol consisting of three hallmark tasks (action recognition, temporal localization, and spatiotemporal localization), eight datasets well received by the community, and four adaptation methods tailoring a foundation model (FM) for a downstream task.
Structured Video-Language Modeling with Temporal Grouping and Spatial Grounding
Existing video-language pre-training methods primarily focus on instance-level alignment between video clips and captions via global contrastive learning but neglect rich fine-grained local information in both videos and text, which is of importance to downstream tasks requiring temporal localization and semantic reasoning.
Unified Visual Relationship Detection with Vision and Language Models
To address this challenge, we propose UniVRD, a novel bottom-up method for Unified Visual Relationship Detection by leveraging vision and language models (VLMs).
Human-Object Interaction Detection
Relationship Detection
+2
Steering Prototypes with Prompt-tuning for Rehearsal-free Continual Learning
In the context of continual learning, prototypes-as representative class embeddings-offer advantages in memory conservation and the mitigation of catastrophic forgetting.
Hierarchically Self-Supervised Transformer for Human Skeleton Representation Learning
Despite the success of fully-supervised human skeleton sequence modeling, utilizing self-supervised pre-training for skeleton sequence representation learning has been an active field because acquiring task-specific skeleton annotations at large scales is difficult.
Exploiting Unlabeled Data with Vision and Language Models for Object Detection
We propose a novel method that leverages the rich semantics available in recent vision and language models to localize and classify objects in unlabeled images, effectively generating pseudo labels for object detection.
Ranked #15 on
Open Vocabulary Object Detection
on MSCOCO
(using extra training data)
Limits of Semistatic Trading Strategies
We show that pointwise limits of semistatic trading strategies in discrete time are again semistatic strategies.
Martingale Schrödinger Bridges and Optimal Semistatic Portfolios
In a two-period financial market where a stock is traded dynamically and European options at maturity are traded statically, we study the so-called martingale Schr\"odinger bridge Q*; that is, the minimal-entropy martingale measure among all models calibrated to option prices.
Are Multimodal Transformers Robust to Missing Modality?
Based on these findings, we propose a principle method to improve the robustness of Transformer models by automatically searching for an optimal fusion strategy regarding input data.
Global Matching with Overlapping Attention for Optical Flow Estimation
In this paper, inspired by the traditional matching-optimization methods where matching is introduced to handle large displacements before energy-based optimizations, we introduce a simple but effective global matching step before the direct regression and develop a learning-based matching-optimization framework, namely GMFlowNet.
Ranked #4 on
Optical Flow Estimation
on KITTI 2015
Min-Max Latency Optimization Based on Sensed Position State Information in Internet of Vehicles
The dual-function radar communication (DFRC) is an essential technology in Internet of Vehicles (IoV).
COMPOSER: Compositional Reasoning of Group Activity in Videos with Keypoint-Only Modality
Group Activity Recognition detects the activity collectively performed by a group of actors, which requires compositional reasoning of actors and objects.
Ranked #2 on
Group Activity Recognition
on Collective Activity
Out-of-Domain Generalization from a Single Source: An Uncertainty Quantification Approach
We are concerned with a worst-case scenario in model generalization, in the sense that a model aims to perform well on many unseen domains while there is only one single domain available for training.
Improved Transformer for High-Resolution GANs
Attention-based models, exemplified by the Transformer, can effectively model long range dependency, but suffer from the quadratic complexity of self-attention operation, making them difficult to be adopted for high-resolution image generation based on Generative Adversarial Networks (GANs).
Ranked #2 on
Image Generation
on CelebA 256x256
(FID metric)
Nested Hierarchical Transformer: Towards Accurate, Data-Efficient and Interpretable Visual Understanding
Hierarchical structures are popular in recent vision transformers, however, they require sophisticated designs and massive datasets to work well.
Ranked #84 on
Image Classification
on CIFAR-10
More Than Just Attention: Improving Cross-Modal Attentions with Contrastive Constraints for Image-Text Matching
Cross-modal attention mechanisms have been widely applied to the image-text matching task and have achieved remarkable improvements thanks to its capability of learning fine-grained relevance across different modalities.
SMIL: Multimodal Learning with Severely Missing Modality
A common assumption in multimodal learning is the completeness of training data, i. e., full modalities are available in all training examples.
Box Re-Ranking: Unsupervised False Positive Suppression for Domain Adaptive Pedestrian Detection
Therefore, it yields our attention to suppress false positive in each target domain in an unsupervised way.
Learning View-Disentangled Human Pose Representation by Contrastive Cross-View Mutual Information Maximization
To evaluate the power of the learned representations, in addition to the conventional fully-supervised action recognition settings, we introduce a novel task called single-shot cross-view action recognition.
View-Invariant, Occlusion-Robust Probabilistic Embedding for Human Pose
Recognition of human poses and actions is crucial for autonomous systems to interact smoothly with people.
Maximum-Entropy Adversarial Data Augmentation for Improved Generalization and Robustness
In this paper, we propose a novel and effective regularization term for adversarial data augmentation.
Beyond Lexical: A Semantic Retrieval Framework for Textual SearchEngine
Search engine has become a fundamental component in various web and mobile applications.
Knowledge as Priors: Cross-Modal Knowledge Generalization for Datasets without Superior Knowledge
Our key idea is to generalize the distilled cross-modal knowledge learned from a Source dataset, which contains paired examples from both modalities, to the Target dataset by modeling knowledge as priors on parameters of the Student.
Learning to Learn Single Domain Generalization
We are concerned with a worst-case scenario in model generalization, in the sense that a model aims to perform well on many unseen domains while there is only one single domain available for training.
Rethinking Kernel Methods for Node Representation Learning on Graphs
Graph kernels are kernel methods measuring graph similarity and serve as a standard tool for graph classification.
Ranked #8 on
Link Prediction
on Cora
Construct Dynamic Graphs for Hand Gesture Recognition via Spatial-Temporal Attention
We propose a Dynamic Graph-Based Spatial-Temporal Attention (DG-STA) method for hand gesture recognition.
Ranked #3 on
Hand Gesture Recognition
on SHREC 2017
Semantic Graph Convolutional Networks for 3D Human Pose Regression
In this paper, we study the problem of learning Graph Convolutional Networks (GCNs) for regression.
Ranked #26 on
Monocular 3D Human Pose Estimation
on Human3.6M
Short-term Road Traffic Prediction based on Deep Cluster at Large-scale Networks
Therefore, a framework combining with a deep clustering (DeepCluster) module is developed for STTP at largescale networks in this paper.
A Driving Intention Prediction Method Based on Hidden Markov Model for Autonomous Driving
Experimental results show that the HMMs trained with the continuous characterization of mobility features can give a higher prediction accuracy when they are used for predicting driving intentions.
Learning to Forecast and Refine Residual Motion for Image-to-Video Generation
We consider the problem of image-to-video translation, where an input image is translated into an output video containing motions of a single object.
CR-GAN: Learning Complete Representations for Multi-view Generation
Generating multi-view images from a single-view input is an essential yet challenging problem.
Cartoonish sketch-based face editing in videos using identity deformation transfer
We first recover the facial identity and expressions from the video by fitting a face morphable model for each frame.
Bridging Saliency Detection to Weakly Supervised Object Detection Based on Self-paced Curriculum Learning
Weakly-supervised object detection (WOD) is a challenging problems in computer vision.
Ranked #34 on
Weakly Supervised Object Detection
on PASCAL VOC 2007
Object Proposal by Multi-Branch Hierarchical Segmentation
Hierarchical segmentation based object proposal methods have become an important step in modern object detection paradigm.
