Visual Navigation
105 papers with code • 6 benchmarks • 16 datasets
Visual Navigation is the problem of navigating an agent, e.g. a mobile robot, in an environment using camera input only. The agent is given a target image (an image it will see from the target position), and its goal is to move from its current position to the target by applying a sequence of actions, based on the camera observations only.
Source: Vision-based Navigation Using Deep Reinforcement Learning
Benchmarks
These leaderboards are used to track progress in Visual Navigation
| Trend | Dataset | Best Model | Paper | Code | Compare |
|---|---|---|---|---|---|
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
Libraries
Use these libraries to find Visual Navigation models and implementations
Replica
AI2-THOR
SUNCG
R2R
2D-3D-S
AVD
HELP
MINOS
House3D Environment
Talk the Walk
Most implemented papers
Self-Monitoring Navigation Agent via Auxiliary Progress Estimation
chihyaoma/selfmonitoring-agent
•
•
ICLR 2019
The Vision-and-Language Navigation (VLN) task entails an agent following navigational instruction in photo-realistic unknown environments.
Learning Exploration Policies for Navigation
taochenshh/exp4nav
•
•
ICLR 2019
Numerous past works have tackled the problem of task-driven navigation.
Scaling and Benchmarking Self-Supervised Visual Representation Learning
facebookresearch/fair_self_supervision_benchmark
•
•
ICCV 2019
Self-supervised learning aims to learn representations from the data itself without explicit manual supervision.
An Open Source and Open Hardware Deep Learning-powered Visual Navigation Engine for Autonomous Nano-UAVs
Nano-size unmanned aerial vehicles (UAVs), with few centimeters of diameter and sub-10 Watts of total power budget, have so far been considered incapable of running sophisticated visual-based autonomous navigation software without external aid from base-stations, ad-hoc local positioning infrastructure, and powerful external computation servers.
Vision-and-Dialog Navigation
mmurray/cvdn
•
•
To train agents that search an environment for a goal location, we define the Navigation from Dialog History task.
VUSFA:Variational Universal Successor Features Approximator to Improve Transfer DRL for Target Driven Visual Navigation
shamanez/VUSFA-Variational-Universal-Successor-Features-Approximator
•
•
In this paper, we show how novel transfer reinforcement learning techniques can be applied to the complex task of target driven navigation using the photorealistic AI2THOR simulator.
SoundSpaces: Audio-Visual Navigation in 3D Environments
facebookresearch/sound-spaces
•
•
ECCV 2020
Moving around in the world is naturally a multisensory experience, but today's embodied agents are deaf---restricted to solely their visual perception of the environment.
Side-Tuning: A Baseline for Network Adaptation via Additive Side Networks
jozhang97/side-tuning
•
•
ECCV 2020
When training a neural network for a desired task, one may prefer to adapt a pre-trained network rather than starting from randomly initialized weights.
Decoupling Exploration and Exploitation for Meta-Reinforcement Learning without Sacrifices
Learning a new task often requires both exploring to gather task-relevant information and exploiting this information to solve the task.
Visual Navigation in Real-World Indoor Environments Using End-to-End Deep Reinforcement Learning
jkulhanek/robot-visual-navigation
•
•
This precludes the use of the learned policy on a real robot.
