Search Results for author:
Found 31 papers, 13 papers with code
Clustering for Protein Representation Learning
We select the highest-scoring clusters and use their medoid nodes for the next iteration of clustering, until we obtain a hierarchical and informative representation of the protein.
EVA: Zero-shot Accurate Attributes and Multi-Object Video Editing
We find that the crux of the issue stems from the imprecise distribution of attention weights across designated regions, including inaccurate text-to-attribute control and attention leakage.
ProtChatGPT: Towards Understanding Proteins with Large Language Models
The protein first undergoes protein encoders and PLP-former to produce protein embeddings, which are then projected by the adapter to conform with the LLM.
HeadStudio: Text to Animatable Head Avatars with 3D Gaussian Splatting
Specifically, we incorporate the FLAME into both 3D representation and score distillation: 1) FLAME-based 3D Gaussian splatting, driving 3D Gaussian points by rigging each point to a FLAME mesh.
Hand-Centric Motion Refinement for 3D Hand-Object Interaction via Hierarchical Spatial-Temporal Modeling
To address this problem, we propose a data-driven method for coarse motion refinement.
DocMSU: A Comprehensive Benchmark for Document-level Multimodal Sarcasm Understanding
Multimodal Sarcasm Understanding (MSU) has a wide range of applications in the news field such as public opinion analysis and forgery detection.
Building Category Graphs Representation with Spatial and Temporal Attention for Visual Navigation
To this end, an agent needs to 1) learn a piece of certain knowledge about the relations of object categories in the world during training and 2) look for the target object based on the pre-learned object category relations and its moving trajectory in the current unseen environment.
A Reliable Representation with Bidirectional Transition Model for Visual Reinforcement Learning Generalization
Visual reinforcement learning has proven effective in solving control tasks with high-dimensional observations.
FlowZero: Zero-Shot Text-to-Video Synthesis with LLM-Driven Dynamic Scene Syntax
Text-to-video (T2V) generation is a rapidly growing research area that aims to translate the scenes, objects, and actions within complex video text into a sequence of coherent visual frames.
Prior-Free Continual Learning with Unlabeled Data in the Wild
Existing CL methods usually reduce forgetting with task priors, \ie using task identity or a subset of previously seen samples for model training.
Point Contrastive Prediction with Semantic Clustering for Self-Supervised Learning on Point Cloud Videos
Instead of contrasting the representations of clips or frames, in this paper, we propose a unified self-supervised framework by conducting contrastive learning at the point level.
Masked Spatio-Temporal Structure Prediction for Self-supervised Learning on Point Cloud Videos
In this paper, we propose a Masked Spatio-Temporal Structure Prediction (MaST-Pre) method to capture the structure of point cloud videos without human annotations.
DPMix: Mixture of Depth and Point Cloud Video Experts for 4D Action Segmentation
The proposed method, named Mixture of Depth and Point cloud video experts (DPMix), achieved the first place in the 4D Action Segmentation Track of the HOI4D Challenge 2023.
Keyword-Aware Relative Spatio-Temporal Graph Networks for Video Question Answering
The main challenge in video question answering (VideoQA) is to capture and understand the complex spatial and temporal relations between objects based on given questions.
A Study on Differentiable Logic and LLMs for EPIC-KITCHENS-100 Unsupervised Domain Adaptation Challenge for Action Recognition 2023
Our research focuses on the innovative application of a differentiable logic loss in the training to leverage the co-occurrence relations between verb and noun, as well as the pre-trained Large Language Models (LLMs) to generate the logic rules for the adaptation to unseen action labels.
Continual Learning with Strong Experience Replay
Experience Replay (ER) is a simple and effective rehearsal-based strategy, which optimizes the model with current training data and a subset of old samples stored in a memory buffer.
Text to Point Cloud Localization with Relation-Enhanced Transformer
To overcome these two challenges, we propose a unified Relation-Enhanced Transformer (RET) to improve representation discriminability for both point cloud and natural language queries.
STPrivacy: Spatio-Temporal Privacy-Preserving Action Recognition
For the first time, we introduce vision Transformers into PPAR by treating a video as a tubelet sequence, and accordingly design two complementary mechanisms, i. e., sparsification and anonymization, to remove privacy from a spatio-temporal perspective.
PointListNet: Deep Learning on 3D Point Lists
Deep neural networks on regular 1D lists (e. g., natural languages) and irregular 3D sets (e. g., point clouds) have made tremendous achievements.
Can We Solve 3D Vision Tasks Starting from A 2D Vision Transformer?
Vision Transformers (ViTs) have proven to be effective, in solving 2D image understanding tasks by training over large-scale image datasets; and meanwhile as a somehow separate track, in modeling the 3D visual world too such as voxels or point clouds.
SEFormer: Structure Embedding Transformer for 3D Object Detection
Effectively preserving and encoding structure features from objects in irregular and sparse LiDAR points is a key challenge to 3D object detection on point cloud.
PSTNet: Point Spatio-Temporal Convolution on Point Cloud Sequences
Then, a spatial convolution is employed to capture the local structure of points in the 3D space, and a temporal convolution is used to model the dynamics of the spatial regions along the time dimension.
Ranked #3 on
3D Action Recognition
on NTU RGB+D
Self-Supervised Global-Local Structure Modeling for Point Cloud Domain Adaptation With Reliable Voted Pseudo Labels
In this paper, we propose an unsupervised domain adaptation method for deep point cloud representation learning.
Point 4D Transformer Networks for Spatio-Temporal Modeling in Point Cloud Videos
To capture the dynamics in point cloud videos, point tracking is usually employed.
Ranked #4 on
3D Action Recognition
on NTU RGB+D
PointRNN: Point Recurrent Neural Network for Moving Point Cloud Processing
We apply PointRNN, PointGRU and PointLSTM to moving point cloud prediction, which aims to predict the future trajectories of points in a set given their history movements.
Cascaded Revision Network for Novel Object Captioning
By this novel cascaded captioning-revising mechanism, CRN can accurately describe images with unseen objects.
Attract or Distract: Exploit the Margin of Open Set
In this paper, we exploit the semantic structure of open set data from two aspects: 1) Semantic Categorical Alignment, which aims to achieve good separability of target known classes by categorically aligning the centroid of target with the source.
Adaptive Exploration for Unsupervised Person Re-Identification
However, a problem of the adaptive selection is that, when an image has too many neighborhoods, it is more likely to attract other images as its neighborhoods.
Cubic LSTMs for Video Prediction
Predicting future frames in videos has become a promising direction of research for both computer vision and robot learning communities.
Complex Event Detection by Identifying Reliable Shots From Untrimmed Videos
relevant) to the given event class, we formulate this task as a multi-instance learning (MIL) problem by taking each video as a bag and the video shots in each video as instances.
Unsupervised Person Re-identification: Clustering and Fine-tuning
Progressively, pedestrian clustering and the CNN model are improved simultaneously until algorithm convergence.
Ranked #12 on
Unsupervised Person Re-Identification
on DukeMTMC-reID
