Search Results for author:
Found 33 papers, 8 papers with code
Towards Multimodal In-Context Learning for Vision & Language Models
Inspired by the emergence of Large Language Models (LLMs) that can truly understand human language, significant progress has been made in aligning other, non-language, modalities to be `understandable' by an LLM, primarily via converting their samples into a sequence of embedded language-like tokens directly fed into the LLM (decoder) input stream.
Efficient Rehearsal Free Zero Forgetting Continual Learning using Adaptive Weight Modulation
This challenge arises due to the tendency of previously learned weights to be adjusted to suit the objectives of new tasks, resulting in a phenomenon called catastrophic forgetting.
Top-Down Network Combines Back-Propagation with Attention
For example, during multi-task learning, the same top-down network is being used for both learning, via propagating feedback signals, and at the same time also for top-down attention, by guiding the bottom-up network to perform a selected task.
Teaching Structured Vision & Language Concepts to Vision & Language Models
Vision and Language (VL) models have demonstrated remarkable zero-shot performance in a variety of tasks.
Teaching Structured Vision&Language Concepts to Vision&Language Models
Vision and Language (VL) models have demonstrated remarkable zero-shot performance in a variety of tasks.
A model for full local image interpretation
We describe a computational model of humans' ability to provide a detailed interpretation of components in a scene.
Image interpretation by iterative bottom-up top-down processing
Scene understanding requires the extraction and representation of scene components together with their properties and inter-relations.
Detector-Free Weakly Supervised Grounding by Separation
In this work, we focus on the task of Detector-Free WSG (DF-WSG) to solve WSG without relying on a pre-trained detector.
Ranked #1 on
Phrase Grounding
on Visual Genome
What can human minimal videos tell us about dynamic recognition models?
In human vision objects and their parts can be visually recognized from purely spatial or purely temporal information but the mechanisms integrating space and time are poorly understood.
What takes the brain so long: Object recognition at the level of minimal images develops for up to seconds of presentation time
Subjects were assigned to one of nine exposure conditions: 200, 500, 1000, 2000 ms with or without masking, as well as unlimited time.
Multi-Task Learning by a Top-Down Control Network
As the range of tasks performed by a general vision system expands, executing multiple tasks accurately and efficiently in a single network has become an important and still open problem.
Task-Based Top-Down Modulation Network for Multi-Task-Learning Applications
Recent approaches address this problem by a channel-wise modulation of the feature-maps along the shared backbone, with task specific vectors, manually or dynamically tuned.
The Cakewalk Method
Notably, we show in this benchmark that fixing the distribution of the surrogate is key to consistently recovering locally optimal solutions, and that our surrogate objective leads to an algorithm that outperforms other methods we have tested in a number of measures.
Efficient Coarse-to-Fine Non-Local Module for the Detection of Small Objects
An image is not just a collection of objects, but rather a graph where each object is related to other objects through spatial and semantic relations.
VQA with no questions-answers training
Methods for teaching machines to answer visual questions have made significant progress in recent years, but current methods still lack important human capabilities, including integrating new visual classes and concepts in a modular manner, providing explanations for the answers and handling new domains without explicit examples.
Understand, Compose and Respond - Answering Visual Questions by a Composition of Abstract Procedures
An image related question defines a specific visual task that is required in order to produce an appropriate answer.
Discovery and usage of joint attention in images
Second, we use a human study to demonstrate the sensitivity of humans to joint attention, suggesting that the detection of such a configuration in an image can be useful for understanding the image, including the goals of the agents and their joint activity, and therefore can contribute to image captioning and related tasks.
Large Field and High Resolution: Detecting Needle in Haystack
The growing use of convolutional neural networks (CNN) for a broad range of visual tasks, including tasks involving fine details, raises the problem of applying such networks to a large field of view, since the amount of computations increases significantly with the number of pixels.
Cakewalk Sampling
We study the task of finding good local optima in combinatorial optimization problems.
A model for interpreting social interactions in local image regions
Understanding social interactions (such as 'hug' or 'fight') is a basic and important capacity of the human visual system, but a challenging and still open problem for modeling.
Structured learning and detailed interpretation of minimal object images
We model the process of human full interpretation of object images, namely the ability to identify and localize all semantic features and parts that are recognized by human observers.
Measuring and modeling the perception of natural and unconstrained gaze in humans and machines
How accurate are humans in determining the gaze direction of others in lifelike scenes, when they can move their heads and eyes freely, and what are the sources of information for the underlying perceptual processes?
Discovering containment: from infants to machines
Current artificial learning systems can recognize thousands of visual categories, or play Go at a champion"s level, but cannot explain infants learning, in particular the ability to learn complex concepts without guidance, in a specific order.
Action Classification via Concepts and Attributes
Classes in natural images tend to follow long tail distributions.
Human Pose Estimation using Deep Consensus Voting
In this paper we consider the problem of human pose estimation from a single still image.
Ranked #37 on
Pose Estimation
on MPII Human Pose
Do You See What I Mean? Visual Resolution of Linguistic Ambiguities
Understanding language goes hand in hand with the ability to integrate complex contextual information obtained via perception.
Visual Concept Recognition and Localization via Iterative Introspection
Convolutional neural networks have been shown to develop internal representations, which correspond closely to semantically meaningful objects and parts, although trained solely on class labels.
Face-space Action Recognition by Face-Object Interactions
Action recognition in still images has seen major improvement in recent years due to advances in human pose estimation, object recognition and stronger feature representations.
Hand-Object Interaction and Precise Localization in Transitive Action Recognition
In this paper we demonstrate how recognition is improved by obtaining precise localization of the action-object and consequently extracting details of the object shape together with the actor-object interaction.
Learning Local Invariant Mahalanobis Distances
For many tasks and data types, there are natural transformations to which the data should be invariant or insensitive.
When Computer Vision Gazes at Cognition
(1) Human accuracy of discriminating targets 8{\deg}-10{\deg} of visual angle apart is around 40% in a free looking gaze task; (2) The ability to interpret gaze of different lookers vary dramatically; (3) This variance can be captured by the computational model; (4) Human outperforms the current model significantly.
Graph Approximation and Clustering on a Budget
We consider the problem of learning from a similarity matrix (such as spectral clustering and lowd imensional embedding), when computing pairwise similarities are costly, and only a limited number of entries can be observed.
Using body-anchored priors for identifying actions in single images
The task is easy for humans but difficult for current approaches to object recognition, because action instances may be similar in terms of body pose, and often require detailed examination of relations between participating objects and body parts in order to be recognized.
