Visual Navigation
105 papers with code • 6 benchmarks • 16 datasets
Visual Navigation is the problem of navigating an agent, e.g. a mobile robot, in an environment using camera input only. The agent is given a target image (an image it will see from the target position), and its goal is to move from its current position to the target by applying a sequence of actions, based on the camera observations only.
Source: Vision-based Navigation Using Deep Reinforcement Learning
Benchmarks
These leaderboards are used to track progress in Visual Navigation
| Trend | Dataset | Best Model | Paper | Code | Compare |
|---|---|---|---|---|---|
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
Libraries
Use these libraries to find Visual Navigation models and implementations
Replica
AI2-THOR
SUNCG
R2R
2D-3D-S
AVD
HELP
MINOS
House3D Environment
Talk the Walk
Most implemented papers
Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environments
peteanderson80/Matterport3DSimulator
•
•
CVPR 2018
This is significant because a robot interpreting a natural-language navigation instruction on the basis of what it sees is carrying out a vision and language process that is similar to Visual Question Answering.
Cognitive Mapping and Planning for Visual Navigation
tensorflow/models
•
•
CVPR 2017
The accumulated belief of the world enables the agent to track visited regions of the environment.
Think Locally, Act Globally: Federated Learning with Local and Global Representations
To this end, we propose a new federated learning algorithm that jointly learns compact local representations on each device and a global model across all devices.
A 64mW DNN-based Visual Navigation Engine for Autonomous Nano-Drones
As part of our general methodology we discuss the software mapping techniques that enable the state-of-the-art deep convolutional neural network presented in [1] to be fully executed on-board within a strict 6 fps real-time constraint with no compromise in terms of flight results, while all processing is done with only 64 mW on average.
Visual Representations for Semantic Target Driven Navigation
We propose to using high level semantic and contextual features including segmentation and detection masks obtained by off-the-shelf state-of-the-art vision as observations and use deep network to learn the navigation policy.
The Regretful Agent: Heuristic-Aided Navigation through Progress Estimation
chihyaoma/regretful-agent
•
•
CVPR 2019
As deep learning continues to make progress for challenging perception tasks, there is increased interest in combining vision, language, and decision-making.
Sim2Real Predictivity: Does Evaluation in Simulation Predict Real-World Performance?
facebookresearch/habitat-api
•
•
Second, we investigate the sim2real predictivity of Habitat-Sim for PointGoal navigation.
POPGym: Benchmarking Partially Observable Reinforcement Learning
proroklab/popgym
•
•
Real world applications of Reinforcement Learning (RL) are often partially observable, thus requiring memory.
Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learning
shamanez/Target-Driven-Visual-Navigation-with-Distributed-PPO
•
•
To address the second issue, we propose AI2-THOR framework, which provides an environment with high-quality 3D scenes and physics engine.
Learning to Learn How to Learn: Self-Adaptive Visual Navigation Using Meta-Learning
In this paper we study the problem of learning to learn at both training and test time in the context of visual navigation.
