no code implementations • ECCV 2020 • Ehsan Elhamifar, Dat Huynh
We address the problem of unsupervised procedure learning from instructional videos of multiple tasks using Deep Neural Networks (DNNs).
no code implementations • 28 Aug 2023 • Zijia Lu, Ehsan Elhamifar
We address the task of supervised action segmentation which aims to partition a video into non-overlapping segments, each representing a different action.
Ranked #2 on Action Segmentation on Breakfast
no code implementations • 21 Jun 2023 • YuHan Shen, Linjie Yang, Longyin Wen, Haichao Yu, Ehsan Elhamifar, Heng Wang
Recent focus in video captioning has been on designing architectures that can consume both video and text modalities, and using large-scale video datasets with text transcripts for pre-training, such as HowTo100M.
Automatic Speech Recognition Automatic Speech Recognition (ASR) +2
no code implementations • 11 Jul 2022 • Hassan Mahmood, Ehsan Elhamifar
We show that the naive extensions of multi-class attacks to the multi-label setting lead to violating label relationships, modeled by a knowledge graph, and can be detected using a consistency verification scheme.
1 code implementation • CVPR 2022 • Zijia Lu, Ehsan Elhamifar
We address the problem of set-supervised action learning, whose goal is to learn an action segmentation model using weak supervision in the form of sets of actions occurring in training videos.
Pseudo Label Weakly Supervised Action Segmentation (Action Set))
1 code implementation • CVPR 2022 • YuHan Shen, Ehsan Elhamifar
To compute the SRE loss, we develop a flexible transcript prediction (FTP) method that uses the output of the action classifier to find both the length of the transcript and the sequence of actions occurring in an unlabeled video.
1 code implementation • CVPR 2022 • Dat Huynh, Jason Kuen, Zhe Lin, Jiuxiang Gu, Ehsan Elhamifar
To address this, we propose a cross-modal pseudo-labeling framework, which generates training pseudo masks by aligning word semantics in captions with visual features of object masks in images.
no code implementations • CVPR 2021 • Nasim Shafiee, Taskin Padir, Ehsan Elhamifar
Predicting human trajectories is an important component of autonomous moving platforms, such as social robots and self-driving cars.
1 code implementation • CVPR 2021 • YuHan Shen, Lu Wang, Ehsan Elhamifar
We address the problem of unsupervised localization of key-steps and feature learning in instructional videos using both visual and language instructions.
no code implementations • 21 May 2021 • Dat Huynh, Ehsan Elhamifar
In addition, instead of building holistic features for classes, we use our attribute features to form dense representations capable of capturing fine-grained attribute details of classes.
Ranked #2 on Zero-Shot Learning on CUB-200-2011
1 code implementation • ICCV 2021 • Zijia Lu, Ehsan Elhamifar
We address the problem of learning to segment actions from weakly-annotated videos, i. e., videos accompanied by transcripts (ordered list of actions).
1 code implementation • ICCV 2021 • Dat Huynh, Ehsan Elhamifar
We study the problem of multi-label zero-shot recognition in which labels are in the form of human-object interactions (combinations of actions on objects), each image may contain multiple interactions and some interactions do not have training images.
Human-Object Interaction Detection Multi-label zero-shot learning +1
no code implementations • NeurIPS 2020 • Dat Huynh, Ehsan Elhamifar
We propose a feature composition framework that learns to extract attribute-based features from training samples and combines them to construct fine-grained features for unseen classes.
no code implementations • 23 Feb 2020 • Setareh Ariafar, Zelda Mariet, Ehsan Elhamifar, Dana Brooks, Jennifer Dy, Jasper Snoek
Casting hyperparameter search as a multi-task Bayesian optimization problem over both hyperparameters and importance sampling design achieves the best of both worlds: by learning a parameterization of IS that trades-off evaluation complexity and quality, we improve upon Bayesian optimization state-of-the-art runtime and final validation error across a variety of datasets and complex neural architectures.
no code implementations • NeurIPS 2019 • Chengguang Xu, Ehsan Elhamifar
We develop a supervised subset selection framework, based on the facility location utility function, which learns to map datasets to their ground-truth representatives.
no code implementations • ICCV 2019 • Ehsan Elhamifar, Zwe Naing
To extract procedure key-steps, we develop an optimization framework that finds a sequence of a small number of states that well represents all videos and is compatible with the state transition model.
no code implementations • NeurIPS 2017 • Ehsan Elhamifar, M. Clara De Paolis Kaluza
In this paper, we develop a new framework for sequential subset selection that finds a set of representatives compatible with the dynamic models of data.
no code implementations • CVPR 2017 • Ehsan Elhamifar, M. Clara De Paolis Kaluza
As the proposed optimization is, in general, NP-hard and non-convex, we study a greedy approach based on unconstrained submodular optimization and also propose an efficient convex relaxation.
no code implementations • NeurIPS 2016 • Ehsan Elhamifar
We propose efficient algorithms for simultaneous clustering and completion of incomplete high-dimensional data that lie in a union of low-dimensional subspaces.
no code implementations • 23 Dec 2014 • Ehsan Elhamifar, Mahdi Soltanolkotabi, Shankar Sastry
High-dimensional data often lie in low-dimensional subspaces corresponding to different classes they belong to.
no code implementations • 25 Jul 2014 • Ehsan Elhamifar, Guillermo Sapiro, S. Shankar Sastry
The solution of our optimization finds representatives and the assignment of each element of the target set to each representative, hence, obtaining a clustering.
no code implementations • 11 Jan 2013 • Mahdi Soltanolkotabi, Ehsan Elhamifar, Emmanuel J. Candès
Subspace clustering refers to the task of finding a multi-subspace representation that best fits a collection of points taken from a high-dimensional space.
no code implementations • NeurIPS 2012 • Ehsan Elhamifar, Guillermo Sapiro, René Vidal
Given pairwise dissimilarities between data points, we consider the problem of finding a subset of data points called representatives or exemplars that can efficiently describe the data collection.
4 code implementations • 5 Mar 2012 • Ehsan Elhamifar, Rene Vidal
In this paper, we propose and study an algorithm, called Sparse Subspace Clustering (SSC), to cluster data points that lie in a union of low-dimensional subspaces.
Ranked #4 on Motion Segmentation on Hopkins155
no code implementations • NeurIPS 2011 • Ehsan Elhamifar, René Vidal
We propose an algorithm called Sparse Manifold Clustering and Embedding (SMCE) for simultaneous clustering and dimensionality reduction of data lying in multiple nonlinear manifolds.