Search Results for author:
Found 48 papers, 26 papers with code
Inference via Interpolation: Contrastive Representations Provably Enable Planning and Inference
The key idea is to apply a variant of contrastive learning to time series data.
Closing the Gap between TD Learning and Supervised Learning -- A Generalisation Point of View
Based on this analysis, we construct new datasets to explicitly test for this property, revealing that SL-based methods lack this stitching property and hence fail to perform combinatorial generalization.
Bridging State and History Representations: Understanding Self-Predictive RL
These findings culminate in a set of preliminary guidelines for RL practitioners.
Contrastive Difference Predictive Coding
Predicting and reasoning about the future lie at the heart of many time-series questions.
A Connection between One-Step Regularization and Critic Regularization in Reinforcement Learning
One-step methods perform regularization by doing just a single step of policy improvement, while critic regularization methods do many steps of policy improvement with a regularized objective.
Contrastive Example-Based Control
In this paper, we propose a method for offline, example-based control that learns an implicit model of multi-step transitions, rather than a reward function.
HIQL: Offline Goal-Conditioned RL with Latent States as Actions
This structure can be very useful, as assessing the quality of actions for nearby goals is typically easier than for more distant goals.
Game-Theoretic Robust Reinforcement Learning Handles Temporally-Coupled Perturbations
To tackle this challenge, we propose GRAD, a novel game-theoretic approach that treats the temporally-coupled robust RL problem as a partially observable two-player zero-sum game.
When Do Transformers Shine in RL? Decoupling Memory from Credit Assignment
The Transformer architecture has been very successful to solve problems that involve long-term dependencies, including in the RL domain.
Stabilizing Contrastive RL: Techniques for Robotic Goal Reaching from Offline Data
Robotic systems that rely primarily on self-supervised learning have the potential to decrease the amount of human annotation and engineering effort required to learn control strategies.
Bitrate-Constrained DRO: Beyond Worst Case Robustness To Unknown Group Shifts
Some robust training algorithms (e. g., Group DRO) specialize to group shifts and require group information on all training points.
Learning Options via Compression
Identifying statistical regularities in solutions to some tasks in multi-task reinforcement learning can accelerate the learning of new tasks.
Contrastive Value Learning: Implicit Models for Simple Offline RL
In this paper, we propose Contrastive Value Learning (CVL), which learns an implicit, multi-step model of the environment dynamics.
Simplifying Model-based RL: Learning Representations, Latent-space Models, and Policies with One Objective
In this work, we propose a single objective which jointly optimizes a latent-space model and policy to achieve high returns while remaining self-consistent.
Contrastive Learning as Goal-Conditioned Reinforcement Learning
While deep RL should automatically acquire such good representations, prior work often finds that learning representations in an end-to-end fashion is unstable and instead equip RL algorithms with additional representation learning parts (e. g., auxiliary losses, data augmentation).
Imitating Past Successes can be Very Suboptimal
Prior work has proposed a simple strategy for reinforcement learning (RL): label experience with the outcomes achieved in that experience, and then imitate the relabeled experience.
Adversarial Unlearning: Reducing Confidence Along Adversarial Directions
Supervised learning methods trained with maximum likelihood objectives often overfit on training data.
RvS: What is Essential for Offline RL via Supervised Learning?
Recent work has shown that supervised learning alone, without temporal difference (TD) learning, can be remarkably effective for offline RL.
C-Planning: An Automatic Curriculum for Learning Goal-Reaching Tasks
Goal-conditioned reinforcement learning (RL) can solve tasks in a wide range of domains, including navigation and manipulation, but learning to reach distant goals remains a central challenge to the field.
Recurrent Model-Free RL Can Be a Strong Baseline for Many POMDPs
However, prior work has found that such recurrent model-free RL methods tend to perform worse than more specialized algorithms that are designed for specific types of POMDPs.
Mismatched No More: Joint Model-Policy Optimization for Model-Based RL
Many model-based reinforcement learning (RL) methods follow a similar template: fit a model to previously observed data, and then use data from that model for RL or planning.
Model-based Reinforcement Learning
Reinforcement Learning (RL)
The Information Geometry of Unsupervised Reinforcement Learning
In this work, we show that unsupervised skill discovery algorithms based on mutual information maximization do not learn skills that are optimal for every possible reward function.
The Essential Elements of Offline RL via Supervised Learning
These methods, which we collectively refer to as reinforcement learning via supervised learning (RvS), involve a number of design decisions, such as policy architectures and how the conditioning variable is constructed.
Robust Predictable Control
Many of the challenges facing today's reinforcement learning (RL) algorithms, such as robustness, generalization, transfer, and computational efficiency are closely related to compression.
Actionable Models: Unsupervised Offline Reinforcement Learning of Robotic Skills
We consider the problem of learning useful robotic skills from previously collected offline data without access to manually specified rewards or additional online exploration, a setting that is becoming increasingly important for scaling robot learning by reusing past robotic data.
Rapid Exploration for Open-World Navigation with Latent Goal Models
We describe a robotic learning system for autonomous exploration and navigation in diverse, open-world environments.
Replacing Rewards with Examples: Example-Based Policy Search via Recursive Classification
Can we devise RL algorithms that instead enable users to specify tasks simply by providing examples of successful outcomes?
Maximum Entropy RL (Provably) Solves Some Robust RL Problems
Many potential applications of reinforcement learning (RL) require guarantees that the agent will perform well in the face of disturbances to the dynamics or reward function.
Model-Based Visual Planning with Self-Supervised Functional Distances
In our experiments, we find that our method can successfully learn models that perform a variety of tasks at test-time, moving objects amid distractors with a simulated robotic arm and even learning to open and close a drawer using a real-world robot.
ViNG: Learning Open-World Navigation with Visual Goals
We propose a learning-based navigation system for reaching visually indicated goals and demonstrate this system on a real mobile robot platform.
C-Learning: Learning to Achieve Goals via Recursive Classification
This problem, which can be viewed as a reframing of goal-conditioned reinforcement learning (RL), is centered around learning a conditional probability density function over future states.
f-IRL: Inverse Reinforcement Learning via State Marginal Matching
Our method outperforms adversarial imitation learning methods in terms of sample efficiency and the required number of expert trajectories on IRL benchmarks.
Learning to be Safe: Deep RL with a Safety Critic
Safety is an essential component for deploying reinforcement learning (RL) algorithms in real-world scenarios, and is critical during the learning process itself.
Interactive Visualization for Debugging RL
Visualization tools for supervised learning allow users to interpret, introspect, and gain an intuition for the successes and failures of their models.
Off-Dynamics Reinforcement Learning: Training for Transfer with Domain Classifiers
Building off of a probabilistic view of RL, we formally show that we can achieve this goal by compensating for the difference in dynamics by modifying the reward function.
Weakly-Supervised Reinforcement Learning for Controllable Behavior
Reinforcement learning (RL) is a powerful framework for learning to take actions to solve tasks.
Rewriting History with Inverse RL: Hindsight Inference for Policy Improvement
In this paper, we show that hindsight relabeling is inverse RL, an observation that suggests that we can use inverse RL in tandem for RL algorithms to efficiently solve many tasks.
Learning to Reach Goals via Iterated Supervised Learning
Current reinforcement learning (RL) algorithms can be brittle and difficult to use, especially when learning goal-reaching behaviors from sparse rewards.
Multi-Goal Reinforcement Learning
Reinforcement Learning (RL)
If MaxEnt RL is the Answer, What is the Question?
In particular, we show (1) that MaxEnt RL can be used to solve a certain class of POMDPs, and (2) that MaxEnt RL is equivalent to a two-player game where an adversary chooses the reward function.
Learning to Reach Goals Without Reinforcement Learning
By maximizing the likelihood of good actions provided by an expert demonstrator, supervised imitation learning can produce effective policies without the algorithmic complexities and optimization challenges of reinforcement learning, at the cost of requiring an expert demonstrator -- typically a person -- to provide the demonstrations.
Avoiding Negative Side-Effects and Promoting Safe Exploration with Imaginative Planning
Our imaginative module can be seen as a ``plug-and-play'' approach to ensuring safety, as it is compatible with any existing RL algorithm and any task with discrete action space.
Efficient Exploration via State Marginal Matching
The SMM objective can be viewed as a two-player, zero-sum game between a state density model and a parametric policy, an idea that we use to build an algorithm for optimizing the SMM objective.
Search on the Replay Buffer: Bridging Planning and Reinforcement Learning
We introduce a general control algorithm that combines the strengths of planning and reinforcement learning to effectively solve these tasks.
Unsupervised Meta-Learning for Reinforcement Learning
In the context of reinforcement learning, meta-learning algorithms acquire reinforcement learning procedures to solve new problems more efficiently by utilizing experience from prior tasks.
Self-Consistent Trajectory Autoencoder: Hierarchical Reinforcement Learning with Trajectory Embeddings
We show that we can learn continuous latent representations of trajectories, which are effective in solving temporally extended and multi-stage problems.
Hierarchical Reinforcement Learning
reinforcement-learning
+2
Diversity is All You Need: Learning Skills without a Reward Function
On a variety of simulated robotic tasks, we show that this simple objective results in the unsupervised emergence of diverse skills, such as walking and jumping.
Leave no Trace: Learning to Reset for Safe and Autonomous Reinforcement Learning
In this work, we propose an autonomous method for safe and efficient reinforcement learning that simultaneously learns a forward and reset policy, with the reset policy resetting the environment for a subsequent attempt.
Who is Mistaken?
We then create a representation of characters' beliefs for two tasks in human action understanding: predicting who is mistaken, and when they are mistaken.
