Search Results for author:
Found 37 papers, 14 papers with code
GOAT-Bench: A Benchmark for Multi-Modal Lifelong Navigation
The Embodied AI community has made significant strides in visual navigation tasks, exploring targets from 3D coordinates, objects, language descriptions, and images.
Mixtral of Experts
In particular, Mixtral vastly outperforms Llama 2 70B on mathematics, code generation, and multilingual benchmarks.
Ranked #9 on
Question Answering
on PIQA
Habitat 3.0: A Co-Habitat for Humans, Avatars and Robots
We present Habitat 3. 0: a simulation platform for studying collaborative human-robot tasks in home environments.
Mistral 7B
We introduce Mistral 7B v0. 1, a 7-billion-parameter language model engineered for superior performance and efficiency.
Ranked #4 on
Zero-Shot Video Question Answer
on NExT-GQA
HomeRobot: Open-Vocabulary Mobile Manipulation
HomeRobot (noun): An affordable compliant robot that navigates homes and manipulates a wide range of objects in order to complete everyday tasks.
Galactic: Scaling End-to-End Reinforcement Learning for Rearrangement at 100k Steps-Per-Second
Specifically, a Fetch robot (equipped with a mobile base, 7DoF arm, RGBD camera, egomotion, and onboard sensing) is spawned in a home environment and asked to rearrange objects - by navigating to an object, picking it up, navigating to a target location, and then placing the object at the target location.
AutoNeRF: Training Implicit Scene Representations with Autonomous Agents
Empirical results show that NeRFs can be trained on actively collected data using just a single episode of experience in an unseen environment, and can be used for several downstream robotic tasks, and that modular trained exploration models outperform other classical and end-to-end baselines.
Navigating to Objects Specified by Images
Our modular method solves sub-tasks of exploration, goal instance re-identification, goal localization, and local navigation.
Navigating to Objects in the Real World
In contrast, end-to-end learning does not, dropping from 77% simulation to 23% real-world success rate due to a large image domain gap between simulation and reality.
Instance-Specific Image Goal Navigation: Training Embodied Agents to Find Object Instances
We consider the problem of embodied visual navigation given an image-goal (ImageNav) where an agent is initialized in an unfamiliar environment and tasked with navigating to a location 'described' by an image.
Retrospectives on the Embodied AI Workshop
We present a retrospective on the state of Embodied AI research.
Habitat-Matterport 3D Semantics Dataset
The scale, quality, and diversity of object annotations far exceed those of prior datasets.
Multi-skill Mobile Manipulation for Object Rearrangement
To tackle the entire task, prior work chains multiple stationary manipulation skills with a point-goal navigation skill, which are learned individually on subtasks.
PONI: Potential Functions for ObjectGoal Navigation with Interaction-free Learning
We propose Potential functions for ObjectGoal Navigation with Interaction-free learning (PONI), a modular approach that disentangles the skills of `where to look?'
Differentiable Spatial Planning using Transformers
We consider the problem of spatial path planning.
SEAL: Self-supervised Embodied Active Learning using Exploration and 3D Consistency
The observations gathered by this exploration policy are labelled using 3D consistency and used to improve the perception model.
Recognizing Scenes from Novel Viewpoints
Humans can perceive scenes in 3D from a handful of 2D views.
FILM: Following Instructions in Language with Modular Methods
In contrast, we propose a modular method with structured representations that (1) builds a semantic map of the scene and (2) performs exploration with a semantic search policy, to achieve the natural language goal.
Building Intelligent Autonomous Navigation Agents
In the first part of the thesis, we discuss our work on short-term navigation using end-to-end reinforcement learning to tackle challenges such as obstacle avoidance, semantic perception, language grounding, and reasoning.
Unsupervised Domain Adaptation for Visual Navigation
We further show that our method can be used to transfer the navigation policies learned in simulation to the real world.
Planning with Submodular Objective Functions
We study planning with submodular objective functions, where instead of maximizing the cumulative reward, the goal is to maximize the objective value induced by a submodular function.
Object Goal Navigation using Goal-Oriented Semantic Exploration
We propose a modular system called, `Goal-Oriented Semantic Exploration' which builds an episodic semantic map and uses it to explore the environment efficiently based on the goal object category.
Ranked #4 on
Robot Navigation
on Habitat 2020 Object Nav test-std
Semantic Curiosity for Active Visual Learning
Instead, we explore a self-supervised approach for training our exploration policy by introducing a notion of semantic curiosity.
Neural Topological SLAM for Visual Navigation
This paper studies the problem of image-goal navigation which involves navigating to the location indicated by a goal image in a novel previously unseen environment.
Learning to Explore using Active Neural SLAM
The use of learning provides flexibility with respect to input modalities (in the SLAM module), leverages structural regularities of the world (in global policies), and provides robustness to errors in state estimation (in local policies).
Cross-Task Knowledge Transfer for Visually-Grounded Navigation
Recent efforts on training visual navigation agents conditioned on language using deep reinforcement learning have been successful in learning policies for two different tasks: learning to follow navigational instructions and embodied question answering.
Embodied Multimodal Multitask Learning
In this paper, we propose a multitask model capable of jointly learning these multimodal tasks, and transferring knowledge of words and their grounding in visual objects across the tasks.
On Evaluation of Embodied Navigation Agents
Skillful mobile operation in three-dimensional environments is a primary topic of study in Artificial Intelligence.
Texar: A Modularized, Versatile, and Extensible Toolbox for Text Generation
The features make Texar particularly suitable for technique sharing and generalization across different text generation applications.
Learning Cognitive Models using Neural Networks
Secondly, for domains where a cognitive model is available, we show that representations learned through CogRL can be used to get accurate estimates of skill difficulty and learning rate parameters without using any student performance data.
Gated Path Planning Networks
Value Iteration Networks (VINs) are effective differentiable path planning modules that can be used by agents to perform navigation while still maintaining end-to-end differentiability of the entire architecture.
Global Pose Estimation with an Attention-based Recurrent Network
The ability for an agent to localize itself within an environment is crucial for many real-world applications.
Active Neural Localization
The results on the 2D environments show the effectiveness of the learned policy in an idealistic setting while results on the 3D environments demonstrate the model's capability of learning the policy and perceptual model jointly from raw-pixel based RGB observations.
Knowledge-based Word Sense Disambiguation using Topic Models
In this paper, we leverage the formalism of topic model to design a WSD system that scales linearly with the number of words in the context.
Ranked #2 on
Word Sense Disambiguation
on Knowledge-based:
Gated-Attention Architectures for Task-Oriented Language Grounding
To perform tasks specified by natural language instructions, autonomous agents need to extract semantically meaningful representations of language and map it to visual elements and actions in the environment.
Playing FPS Games with Deep Reinforcement Learning
Advances in deep reinforcement learning have allowed autonomous agents to perform well on Atari games, often outperforming humans, using only raw pixels to make their decisions.
