About

Benchmarks

TREND DATASET BEST METHOD PAPER TITLE PAPER CODE COMPARE

Datasets

Greatest papers with code

Look Before You Leap: Bridging Model-Free and Model-Based Reinforcement Learning for Planned-Ahead Vision-and-Language Navigation

ECCV 2018 peteanderson80/Matterport3DSimulator

In this paper, we take a radical approach to bridge the gap between synthetic studies and real-world practices---We propose a novel, planned-ahead hybrid reinforcement learning model that combines model-free and model-based reinforcement learning to solve a real-world vision-language navigation task.

ROBOT NAVIGATION VISION AND LANGUAGE NAVIGATION VISION-LANGUAGE NAVIGATION

Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environments

CVPR 2018 peteanderson80/Matterport3DSimulator

This is significant because a robot interpreting a natural-language navigation instruction on the basis of what it sees is carrying out a vision and language process that is similar to Visual Question Answering.

VISION AND LANGUAGE NAVIGATION VISUAL NAVIGATION VISUAL QUESTION ANSWERING

The Regretful Navigation Agent for Vision-and-Language Navigation

CVPR 2019 (Oral) 2019 chihyaoma/regretful-agent

As deep learning continues to make progress for challenging perception tasks, there is increased interest in combining vision, language, and decision-making.

DECISION MAKING VISION AND LANGUAGE NAVIGATION VISION-LANGUAGE NAVIGATION VISUAL NAVIGATION

The Regretful Agent: Heuristic-Aided Navigation through Progress Estimation

CVPR 2019 chihyaoma/regretful-agent

As deep learning continues to make progress for challenging perception tasks, there is increased interest in combining vision, language, and decision-making.

DECISION MAKING VISION AND LANGUAGE NAVIGATION VISION-LANGUAGE NAVIGATION VISUAL NAVIGATION

Speaker-Follower Models for Vision-and-Language Navigation

NeurIPS 2018 ronghanghu/speaker_follower

We use this speaker model to (1) synthesize new instructions for data augmentation and to (2) implement pragmatic reasoning, which evaluates how well candidate action sequences explain an instruction.

DATA AUGMENTATION VISION AND LANGUAGE NAVIGATION

Retouchdown: Adding Touchdown to StreetLearn as a Shareable Resource for Language Grounding Tasks in Street View

10 Jan 2020clic-lab/touchdown

These have been added to the StreetLearn dataset and can be obtained via the same process as used previously for StreetLearn.

VISION AND LANGUAGE NAVIGATION

Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environments

ECCV 2020 jacobkrantz/VLN-CE

We develop a language-guided navigation task set in a continuous 3D environment where agents must execute low-level actions to follow natural language navigation directions.

VISION AND LANGUAGE NAVIGATION

VALAN: Vision and Language Agent Navigation

6 Dec 2019google-research/valan

VALAN is a lightweight and scalable software framework for deep reinforcement learning based on the SEED RL architecture.

VISION AND LANGUAGE NAVIGATION

Tactical Rewind: Self-Correction via Backtracking in Vision-and-Language Navigation

CVPR 2019 Kelym/FAST

We present the Frontier Aware Search with backTracking (FAST) Navigator, a general framework for action decoding, that achieves state-of-the-art results on the Room-to-Room (R2R) Vision-and-Language navigation challenge of Anderson et.

VISION AND LANGUAGE NAVIGATION VISION-LANGUAGE NAVIGATION