CURL: Contrastive Unsupervised Representations for Reinforcement Learning

8 Apr 2020  ·  Aravind Srinivas, Michael Laskin, Pieter Abbeel ·

We present CURL: Contrastive Unsupervised Representations for Reinforcement Learning. CURL extracts high-level features from raw pixels using contrastive learning and performs off-policy control on top of the extracted features. CURL outperforms prior pixel-based methods, both model-based and model-free, on complex tasks in the DeepMind Control Suite and Atari Games showing 1.9x and 1.2x performance gains at the 100K environment and interaction steps benchmarks respectively. On the DeepMind Control Suite, CURL is the first image-based algorithm to nearly match the sample-efficiency of methods that use state-based features. Our code is open-sourced and available at https://github.com/MishaLaskin/curl.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Atari Games 100k Atari 100k CURL Mean Human-Normalized Score 0.381 # 13
Medium Human-Normalized Score 0.175 # 15
Atari Games Atari 2600 Alien CURL Score 1148.2 # 37
Atari Games Atari 2600 Amidar CURL Score 232.3 # 36
Atari Games Atari 2600 Assault CURL Score 543.7 # 43
Atari Games Atari 2600 Asterix CURL Score 524.3 # 48
Atari Games Atari 2600 Bank Heist CURL Score 193.7 # 41
Atari Games Atari 2600 Battle Zone CURL Score 11208 # 44
Atari Games Atari 2600 Boxing CURL Score 4.8 # 45
Atari Games Atari 2600 Breakout CURL Score 18.2 # 53
Atari Games Atari 2600 Chopper Command CURL Score 1198 # 43
Atari Games Atari 2600 Crazy Climber CURL Score 27805.6 # 44
Atari Games Atari 2600 Demon Attack CURL Score 834 # 43
Atari Games Atari 2600 Freeway CURL Score 27.9 # 41
Atari Games Atari 2600 Frostbite CURL Score 924 # 35
Atari Games Atari 2600 Gopher CURL Score 801.4 # 42
Atari Games Atari 2600 HERO CURL Score 6235.1 # 43
Atari Games Atari 2600 James Bond CURL Medium Human-Normalized Score 400.1 # 1
Atari Games Atari 2600 Kangaroo CURL Score 345.3 # 42
Atari Games Atari 2600 Krull CURL Score 3833.6 # 41
Atari Games Atari 2600 Kung-Fu Master CURL Score 14280 # 42
Atari Games Atari 2600 Ms. Pacman CURL Score 1492.8 # 37
Atari Games Atari 2600 Pong CURL Score 2.1 # 49
Atari Games Atari 2600 Private Eye CURL Score 105.2 # 41
Atari Games Atari 2600 Q*Bert CURL Score 1225.6 # 49
Atari Games Atari 2600 Road Runner CURL Score 6786.7 # 41
Atari Games Atari 2600 Seaquest CURL Score 408 # 53
Atari Games Atari 2600 Up and Down CURL Score 2735.2 # 42
Continuous Control Ball in cup, catch (DMControl100k) CURL Score 769 # 1
Continuous Control Ball in cup, catch (DMControl500k) CURL Score 959 # 1
Continuous Control Cartpole, swingup (DMControl100k) CURL Score 582 # 1
Continuous Control Cartpole, swingup (DMControl500k) CURL Score 841 # 1
Continuous Control Cheetah, run (DMControl100k) CURL Score 299 # 1
Continuous Control Cheetah, run (DMControl500k) CURL Score 518 # 1
Continuous Control Finger, spin (DMControl100k) CURL Score 767 # 1
Continuous Control Finger, spin (DMControl500k) CURL Score 926 # 1
Continuous Control Reacher, easy (DMControl100k) CURL Score 538 # 1
Continuous Control Reacher, easy (DMControl500k) CURL Score 929 # 1
Continuous Control Walker, walk (DMControl100k) CURL Score 403 # 1
Continuous Control Walker, walk (DMControl500k) CURL Score 902 # 1

Methods


No methods listed for this paper. Add relevant methods here