Search Results for author:
Found 14 papers, 4 papers with code
Seeing the Unseen: Visual Common Sense for Semantic Placement
Datasets for image description are typically constructed by curating relevant images and asking humans to annotate the contents of the image; neither of those two steps are straightforward for objects not present in the image.
Imitating Shortest Paths in Simulation Enables Effective Navigation and Manipulation in the Real World
Reinforcement learning (RL) with dense rewards and imitation learning (IL) with human-generated trajectories are the most widely used approaches for training modern embodied agents.
Selective Visual Representations Improve Convergence and Generalization for Embodied AI
Inspired by selective attention in humans-the process through which people filter their perception based on their experiences, knowledge, and the task at hand-we introduce a parameter-efficient approach to filter visual stimuli for embodied AI.
Moving Forward by Moving Backward: Embedding Action Impact over Action Semantics
A common assumption when training embodied agents is that the impact of taking an action is stable; for instance, executing the "move ahead" action will always move the agent forward by a fixed distance, perhaps with some small amount of actuator-induced noise.
Pushing it out of the Way: Interactive Visual Navigation
In this paper, we study the problem of interactive navigation where agents learn to change the environment to navigate more efficiently to their goals.
AllenAct: A Framework for Embodied AI Research
The domain of Embodied AI, in which agents learn to complete tasks through interaction with their environment from egocentric observations, has experienced substantial growth with the advent of deep reinforcement learning and increased interest from the computer vision, NLP, and robotics communities.
Style Example-Guided Text Generation using Generative Adversarial Transformers
The style encoder extracts a style code from the reference example, and the text decoder generates texts based on the style code and the context.
Visual Reaction: Learning to Play Catch with Your Drone
In this paper we address the problem of visual reaction: the task of interacting with dynamic environments where the changes in the environment are not necessarily caused by the agent itself.
Omnidirectional CNN for Visual Place Recognition and Navigation
Given a visual input, the task of the O-CNN is not to retrieve the matched place exemplar, but to retrieve the closest place exemplar and estimate the relative distance between the input and the closest place.
Self-view Grounding Given a Narrated 360° Video
The negative log reconstruction loss of the reverse sentence (referred to as "irrelevant loss") is jointly minimized to encourage the reverse sentence to be different from the given sentence.
Visual Forecasting by Imitating Dynamics in Natural Sequences
This allows us to apply IRL at scale and directly imitate the dynamics in high-dimensional continuous visual sequences from the raw pixel values.
Agent-Centric Risk Assessment: Accident Anticipation and Risky Region Localization
For survival, a living agent must have the ability to assess risk (1) by temporally anticipating accidents before they occur, and (2) by spatially localizing risky regions in the environment to move away from threats.
Leveraging Video Descriptions to Learn Video Question Answering
Then, a large number of candidate QA pairs are automatically generated from descriptions rather than manually annotated.
Title Generation for User Generated Videos
Finally, our sentence augmentation method also outperforms the baselines on the M-VAD dataset.
