Search Results for author:
Found 19 papers, 12 papers with code
VideoPrism: A Foundational Visual Encoder for Video Understanding
We introduce VideoPrism, a general-purpose video encoder that tackles diverse video understanding tasks with a single frozen model.
Neurosymbolic Grounding for Compositional World Models
We introduce Cosmos, a framework for object-centric world modeling that is designed for compositional generalization (CG), i. e., high performance on unseen input scenes obtained through the composition of known visual "atoms."
BKinD-3D: Self-Supervised 3D Keypoint Discovery from Multi-View Videos
In this way, we discover keypoints without requiring manual supervision in videos of humans and rats, demonstrating the potential of 3D keypoint discovery for studying behavior.
Neurosymbolic Programming for Science
Neurosymbolic Programming (NP) techniques have the potential to accelerate scientific discovery.
MABe22: A Multi-Species Multi-Task Benchmark for Learned Representations of Behavior
We introduce MABe22, a large-scale, multi-agent video and trajectory benchmark to assess the quality of learned behavior representations.
Deep Neural Imputation: A Framework for Recovering Incomplete Brain Recordings
Neuroscientists and neuroengineers have long relied on multielectrode neural recordings to study the brain.
Open-Source Tools for Behavioral Video Analysis: Setup, Methods, and Development
Recently developed methods for video analysis, especially models for pose estimation and behavior classification, are transforming behavioral quantification to be more precise, scalable, and reproducible in fields such as neuroscience and ethology.
Self-Supervised Keypoint Discovery in Behavioral Videos
We propose a method for learning the posture and structure of agents from unlabelled behavioral videos.
Ranked #1 on
Unsupervised Human Pose Estimation
on Human3.6M
Automatic Synthesis of Diverse Weak Supervision Sources for Behavior Analysis
We evaluate AutoSWAP in three behavior analysis domains and demonstrate that AutoSWAP outperforms existing approaches using only a fraction of the data.
Unsupervised Learning of Neurosymbolic Encoders
We present a framework for the unsupervised learning of neurosymbolic encoders, which are encoders obtained by composing neural networks with symbolic programs from a domain-specific language.
Interpreting Expert Annotation Differences in Animal Behavior
Hand-annotated data can vary due to factors such as subjective differences, intra-rater variability, and differing annotator expertise.
The Multi-Agent Behavior Dataset: Mouse Dyadic Social Interactions
Multi-agent behavior modeling aims to understand the interactions that occur between agents.
Learning View-Disentangled Human Pose Representation by Contrastive Cross-View Mutual Information Maximization
To evaluate the power of the learned representations, in addition to the conventional fully-supervised action recognition settings, we introduce a novel task called single-shot cross-view action recognition.
Task Programming: Learning Data Efficient Behavior Representations
The tasks in our method can be efficiently engineered by domain experts through a process we call "task programming", which uses programs to explicitly encode structured knowledge from domain experts.
View-Invariant, Occlusion-Robust Probabilistic Embedding for Human Pose
Recognition of human poses and actions is crucial for autonomous systems to interact smoothly with people.
Learning Differentiable Programs with Admissible Neural Heuristics
This relaxed program is differentiable and can be trained end-to-end, and the resulting training loss is an approximately admissible heuristic that can guide the combinatorial search.
EEV: A Large-Scale Dataset for Studying Evoked Expressions from Video
The ability to predict evoked affect from a video, before viewers watch the video, can help in content creation and video recommendation.
View-Invariant Probabilistic Embedding for Human Pose
Depictions of similar human body configurations can vary with changing viewpoints.
Ranked #1 on
Pose Retrieval
on MPI-INF-3DHP
GLA in MediaEval 2018 Emotional Impact of Movies Task
Towards a better understanding of viewer impact, we present our methods for the MediaEval 2018 Emotional Impact of Movies Task to predict the expected valence and arousal continuously in movies.
