Search Results for author:
Found 22 papers, 12 papers with code
HOI-Ref: Hand-Object Interaction Referral in Egocentric Vision
Our results demonstrate that VLMs trained for referral on third person images fail to recognise and refer hands and objects in egocentric images.
Are you Struggling? Dataset and Baselines for Struggle Determination in Assembly Videos
Determining when people are struggling from video enables a finer-grained understanding of actions and opens opportunities for building intelligent support visual interfaces.
Video Editing for Video Retrieval
The teacher model is employed to edit the clips in the training set whereas the student model trains on the edited clips.
GenHowTo: Learning to Generate Actions and State Transformations from Instructional Videos
We address the task of generating temporally consistent and physically plausible images of actions and object state transformations.
Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives
We present Ego-Exo4D, a diverse, large-scale multimodal multiview video dataset and benchmark challenge.
Learning Temporal Sentence Grounding From Narrated EgoVideos
Compared to traditional benchmarks on which this task is evaluated, these datasets offer finer-grained sentences to ground in notably longer videos.
Egocentric Video-Language Pretraining @ Ego4D Challenge 2022
In this report, we propose a video-language pretraining (VLP) based solution \cite{kevin2022egovlp} for four Ego4D challenge tasks, including Natural Language Query (NLQ), Moment Query (MQ), Object State Change Classification (OSCC), and PNR Localization (PNR).
Egocentric Video-Language Pretraining
Video-Language Pretraining (VLP), which aims to learn transferable representation to advance a wide range of video-text downstream tasks, has recently received increasing attention.
Domain Adaptation in Multi-View Embedding for Cross-Modal Video Retrieval
Given a gallery of uncaptioned video sequences, this paper considers the task of retrieving videos based on their relevance to an unseen text query.
Ego4D: Around the World in 3,000 Hours of Egocentric Video
We introduce Ego4D, a massive-scale egocentric video dataset and benchmark suite.
On Semantic Similarity in Video Retrieval
Current video retrieval efforts all found their evaluation on an instance-based assumption, that only a single caption is relevant to a query video and vice versa.
Supervision Levels Scale (SLS)
We propose a three-dimensional discrete and incremental scale to encode a method's level of supervision - i. e. the data and labels used when training a model to achieve a given performance.
Rescaling Egocentric Vision
This paper introduces the pipeline to extend the largest dataset in egocentric vision, EPIC-KITCHENS.
Ranked #6 on
Action Anticipation
on EPIC-KITCHENS-100
The EPIC-KITCHENS Dataset: Collection, Challenges and Baselines
Our dataset features 55 hours of video consisting of 11. 5M frames, which we densely labelled for a total of 39. 6K action segments and 454. 2K object bounding boxes.
Fine-Grained Action Retrieval Through Multiple Parts-of-Speech Embeddings
We report the first retrieval results on fine-grained actions for the large-scale EPIC dataset, in a generalised zero-shot setting.
Learning Visual Actions Using Multiple Verb-Only Labels
We collect multi-verb annotations for three action video datasets and evaluate the verb-only labelling representations for action recognition and cross-modal retrieval (video-to-text and text-to-video).
Towards an Unequivocal Representation of Actions
This work introduces verb-only representations for actions and interactions; the problem of describing similar motions (e. g. 'open door', 'open cupboard'), and distinguish differing ones (e. g. 'open door' vs 'open bottle') using verb-only labels.
Scaling Egocentric Vision: The EPIC-KITCHENS Dataset
First-person vision is gaining interest as it offers a unique viewpoint on people's interaction with objects, their attention, and even intention.
Trespassing the Boundaries: Labeling Temporal Bounds for Object Interactions in Egocentric Video
Manual annotations of temporal bounds for object interactions (i. e. start and end times) are typical training input to recognition, localization and detection algorithms.
Improving Classification by Improving Labelling: Introducing Probabilistic Multi-Label Object Interaction Recognition
This work deviates from easy-to-define class boundaries for object interactions.
SEMBED: Semantic Embedding of Egocentric Action Videos
We present SEMBED, an approach for embedding an egocentric object interaction video in a semantic-visual graph to estimate the probability distribution over its potential semantic labels.
