Search Results for author:
Found 76 papers, 22 papers with code
Genie: Generative Interactive Environments
We introduce Genie, the first generative interactive environment trained in an unsupervised manner from unlabelled Internet videos.
Diversifying AI: Towards Creative Chess with AlphaZero
In particular, we investigate whether a team of diverse AI systems can outperform a single AI in challenging tasks by generating more ideas as a group and then selecting the best ones.
A Definition of Continual Reinforcement Learning
Using this new language, we define a continual learning agent as one that can be understood as carrying out an implicit search process indefinitely, and continual reinforcement learning as the setting in which the best agents are all continual learning agents.
On the Convergence of Bounded Agents
Standard models of the reinforcement learning problem give rise to a straightforward definition of convergence: An agent converges when its behavior or performance in each environment state stops changing.
Structured State Space Models for In-Context Reinforcement Learning
We propose a modification to a variant of S4 that enables us to initialise and reset the hidden state in parallel, allowing us to tackle reinforcement learning tasks.
Hierarchical Reinforcement Learning in Complex 3D Environments
Hierarchical Reinforcement Learning (HRL) agents have the potential to demonstrate appealing capabilities such as planning and exploration with abstraction, transfer, and skill reuse.
Hierarchical Reinforcement Learning
reinforcement-learning
+1
ReLOAD: Reinforcement Learning with Optimistic Ascent-Descent for Last-Iterate Convergence in Constrained MDPs
Such applications often require to put constraints on the agent's behavior.
Composing Task Knowledge with Modular Successor Feature Approximators
Recently, the Successor Features and Generalized Policy Improvement (SF&GPI) framework has been proposed as a method for learning, composing, and transferring predictive knowledge and behavior.
POMRL: No-Regret Learning-to-Plan with Increasing Horizons
We study the problem of planning under model uncertainty in an online meta-reinforcement learning (RL) setting where an agent is presented with a sequence of related tasks with limited interactions per task.
Discovering Evolution Strategies via Meta-Black-Box Optimization
Optimizing functions without access to gradients is the remit of black-box methods such as evolution strategies.
Planning to the Information Horizon of BAMDPs via Epistemic State Abstraction
The Bayes-Adaptive Markov Decision Process (BAMDP) formalism pursues the Bayes-optimal solution to the exploration-exploitation trade-off in reinforcement learning.
In-context Reinforcement Learning with Algorithm Distillation
We propose Algorithm Distillation (AD), a method for distilling reinforcement learning (RL) algorithms into neural networks by modeling their training histories with a causal sequence model.
Palm up: Playing in the Latent Manifold for Unsupervised Pretraining
In this work, we aim to bring the best of both worlds and propose an algorithm that exhibits an exploratory behavior whilst it utilizes large diverse datasets.
Meta-Gradients in Non-Stationary Environments
We support these results with a qualitative analysis of resulting meta-parameter schedules and learned functions of context features.
Mastering the Game of Stratego with Model-Free Multiagent Reinforcement Learning
It has the additional complexity of requiring decision-making under imperfect information, similar to Texas hold'em poker, which has a significantly smaller game tree (on the order of $10^{164}$ nodes).
Discovering Policies with DOMiNO: Diversity Optimization Maintaining Near Optimality
Finding different solutions to the same problem is a key aspect of intelligence associated with creativity and adaptation to novel situations.
GrASP: Gradient-Based Affordance Selection for Planning
Our empirical work shows that it is feasible to learn to select both primitive-action and option affordances, and that simultaneously learning to select affordances and planning with a learned value-equivalent model can outperform model-free RL.
On the Expressivity of Markov Reward
We then provide a set of polynomial-time algorithms that construct a Markov reward function that allows an agent to optimize tasks of each of these three types, and correctly determine when no such reward function exists.
Learning to Learn End-to-End Goal-Oriented Dialog From Related Dialog Tasks
For each goal-oriented dialog task of interest, large amounts of data need to be collected for end-to-end learning of a neural dialog system.
Bootstrapped Meta-Learning
We achieve a new state-of-the art for model-free agents on the Atari ALE benchmark and demonstrate that it yields both performance and efficiency gains in multi-task meta-learning.
Proper Value Equivalence
The value-equivalence (VE) principle proposes a simple answer to this question: a model should capture the aspects of the environment that are relevant for value-based planning.
Model-based Reinforcement Learning
Reinforcement Learning (RL)
Discovering Diverse Nearly Optimal Policies with Successor Features
We propose Diverse Successive Policies, a method for discovering policies that are diverse in the space of Successor Features, while assuring that they are near optimal.
Reward is enough for convex MDPs
Maximising a cumulative reward function that is Markov and stationary, i. e., defined over state-action pairs and independent of time, is sufficient to capture many kinds of goals in a Markov decision process (MDP).
Reinforcement Learning of Implicit and Explicit Control Flow in Instructions
Learning to flexibly follow task instructions in dynamic environments poses interesting challenges for reinforcement learning agents.
Discovery of Options via Meta-Learned Subgoals
Temporal abstractions in the form of options have been shown to help reinforcement learning (RL) agents learn faster.
Adaptive Pairwise Weights for Temporal Credit Assignment
In this empirical paper, we explore heuristics based on more general pairwise weightings that are functions of the state in which the action was taken, the state at the time of the reward, as well as the time interval between the two.
Learning State Representations from Random Deep Action-conditional Predictions
Our main contribution in this work is an empirical finding that random General Value Functions (GVFs), i. e., deep action-conditional predictions -- random both in what feature of observations they predict as well as in the sequence of actions the predictions are conditioned upon -- form good auxiliary tasks for reinforcement learning (RL) problems.
Discovering a set of policies for the worst case reward
Our main contribution is a policy iteration algorithm that builds a set of policies in order to maximize the worst-case performance of the resulting SMP on the set of tasks.
Efficient Querying for Cooperative Probabilistic Commitments
Multiagent systems can use commitments as the core of a general coordination infrastructure, supporting both cooperative and non-cooperative interactions.
On Efficiency in Hierarchical Reinforcement Learning
Hierarchical Reinforcement Learning (HRL) approaches promise to provide more efficient solutions to sequential decision making problems, both in terms of statistical as well as computational efficiency.
The Value Equivalence Principle for Model-Based Reinforcement Learning
As our main contribution, we introduce the principle of value equivalence: two models are value equivalent with respect to a set of functions and policies if they yield the same Bellman updates.
Model-based Reinforcement Learning
reinforcement-learning
+2
Reinforcement Learning for Sparse-Reward Object-Interaction Tasks in a First-person Simulated 3D Environment
In this work, we show that one can learn object-interaction tasks from scratch without supervision by learning an attentive object-model as an auxiliary task during task learning with an object-centric relational RL agent.
Discovering Reinforcement Learning Algorithms
Automating the discovery of update rules from data could lead to more efficient algorithms, or algorithms that are better adapted to specific environments.
Meta-Gradient Reinforcement Learning with an Objective Discovered Online
In this work, we propose an algorithm based on meta-gradient descent that discovers its own objective, flexibly parameterised by a deep neural network, solely from interactive experience with its environment.
Learning to Play No-Press Diplomacy with Best Response Policy Iteration
It also features a large combinatorial action space and simultaneous moves, which are challenging for RL algorithms.
A Self-Tuning Actor-Critic Algorithm
Reinforcement learning algorithms are highly sensitive to the choice of hyperparameters, typically requiring significant manual effort to identify hyperparameters that perform well on a new domain.
How Should an Agent Practice?
We present a method for learning intrinsic reward functions to drive the learning of an agent during periods of practice in which extrinsic task rewards are not available.
What Can Learned Intrinsic Rewards Capture?
Furthermore, we show that unlike policy transfer methods that capture "how" the agent should behave, the learned reward functions can generalise to other kinds of agents and to changes in the dynamics of the environment by capturing "what" the agent should strive to do.
Hindsight Credit Assignment
We consider the problem of efficient credit assignment in reinforcement learning.
No-Press Diplomacy: Modeling Multi-Agent Gameplay
Diplomacy is a seven-player non-stochastic, non-cooperative game, where agents acquire resources through a mix of teamwork and betrayal.
Deep Reinforcement Learning for Multi-Driver Vehicle Dispatching and Repositioning Problem
Order dispatching and driver repositioning (also known as fleet management) in the face of spatially and temporally varying supply and demand are central to a ride-sharing platform marketplace.
Disentangled Cumulants Help Successor Representations Transfer to New Tasks
This is in contrast to the state-of-the-art reinforcement learning agents, which typically start learning each new task from scratch and struggle with knowledge transfer.
Object-oriented state editing for HRL
We introduce agents that use object-oriented reasoning to consider alternate states of the world in order to more quickly find solutions to problems.
Sample Complexity of Reinforcement Learning using Linearly Combined Model Ensembles
As an extension, we also consider the more challenging problem of model selection, where the state features are unknown and can be chosen from a large candidate set.
Multiagent Reinforcement Learning in Games with an Iterated Dominance Solution
We show that MARL converges to the desired outcome if the rewards are designed so that exerting effort is the iterated dominance solution, but fails if it is merely a Nash equilibrium.
Discovery of Useful Questions as Auxiliary Tasks
Arguably, intelligent agents ought to be able to discover their own questions so that in learning answers for them they learn unanticipated useful knowledge and skills; this departs from the focus in much of machine learning on agents learning answers to externally defined questions.
No Press Diplomacy: Modeling Multi-Agent Gameplay
Diplomacy is a seven-player non-stochastic, non-cooperative game, where agents acquire resources through a mix of teamwork and betrayal.
Behaviour Suite for Reinforcement Learning
bsuite is a collection of carefully-designed experiments that investigate core capabilities of reinforcement learning (RL) agents with two objectives.
Learning Independently-Obtainable Reward Functions
We present a novel method for learning a set of disentangled reward functions that sum to the original environment reward and are constrained to be independently obtainable.
Generative Adversarial Self-Imitation Learning
This paper explores a simple regularizer for reinforcement learning by proposing Generative Adversarial Self-Imitation Learning (GASIL), which encourages the agent to imitate past good trajectories via generative adversarial imitation learning framework.
Completing State Representations using Spectral Learning
A central problem in dynamical system modeling is state discovery—that is, finding a compact summary of the past that captures the information needed to predict the future.
Learning End-to-End Goal-Oriented Dialog with Multiple Answers
We also propose a new and more effective testbed, permuted-bAbI dialog tasks, by introducing multiple valid next utterances to the original-bAbI dialog tasks, which allows evaluation of goal-oriented dialog systems in a more realistic setting.
Many-Goals Reinforcement Learning
Second, we explore whether many-goals updating can be used to pre-train a network to subsequently learn faster and better on a single main task of interest.
Self-Imitation Learning
This paper proposes Self-Imitation Learning (SIL), a simple off-policy actor-critic algorithm that learns to reproduce the agent's past good decisions.
Ranked #3 on
Atari Games
on Atari 2600 Atlantis
NE-Table: A Neural key-value table for Named Entities
Many Natural Language Processing (NLP) tasks depend on using Named Entities (NEs) that are contained in texts and in external knowledge sources.
On Learning Intrinsic Rewards for Policy Gradient Methods
In this paper we derive a novel algorithm for learning intrinsic rewards for policy-gradient based learning agents.
The Advantage of Doubling: A Deep Reinforcement Learning Approach to Studying the Double Team in the NBA
During the 2017 NBA playoffs, Celtics coach Brad Stevens was faced with a difficult decision when defending against the Cavaliers: "Do you double and risk giving up easy shots, or stay at home and do the best you can?"
A Neural Method for Goal-Oriented Dialog Systems to interact with Named Entities
Many goal-oriented dialog tasks, especially ones in which the dialog system has to interact with external knowledge sources such as databases, have to handle a large number of Named Entities (NEs).
Markov Decision Processes with Continuous Side Information
Because our lower bound has an exponential dependence on the dimension, we consider a tractable linear setting where the context is used to create linear combinations of a finite set of MDPs.
Value Prediction Network
This paper proposes a novel deep reinforcement learning (RL) architecture, called Value Prediction Network (VPN), which integrates model-free and model-based RL methods into a single neural network.
Ranked #9 on
Atari Games
on Atari 2600 Krull
Understanding and Predicting Empathic Behavior in Counseling Therapy
Counselor empathy is associated with better outcomes in psychology and behavioral counseling.
Zero-Shot Task Generalization with Multi-Task Deep Reinforcement Learning
As a step towards developing zero-shot task generalization capabilities in reinforcement learning (RL), we introduce a new RL problem where the agent should learn to execute sequences of instructions after learning useful skills that solve subtasks.
Repeated Inverse Reinforcement Learning
We introduce a novel repeated Inverse Reinforcement Learning problem: the agent has to act on behalf of a human in a sequence of tasks and wishes to minimize the number of tasks that it surprises the human by acting suboptimally with respect to how the human would have acted.
Predicting Counselor Behaviors in Motivational Interviewing Encounters
As the number of people receiving psycho-therapeutic treatment increases, the automatic evaluation of counseling practice arises as an important challenge in the clinical domain.
Minimizing Maximum Regret in Commitment Constrained Sequential Decision Making
In cooperative multiagent planning, it can often be beneficial for an agent to make commitments about aspects of its behavior to others, allowing them in turn to plan their own behaviors without taking the agent's detailed behavior into account.
Control of Memory, Active Perception, and Action in Minecraft
In this paper, we introduce a new set of reinforcement learning (RL) tasks in Minecraft (a flexible 3D world).
Deep Learning for Reward Design to Improve Monte Carlo Tree Search in ATARI Games
We present an adaptation of PGRD (policy-gradient for reward-design) for learning a reward-bonus function to improve UCT (a MCTS algorithm).
Towards Resolving Unidentifiability in Inverse Reinforcement Learning
We first demonstrate that if the learner can experiment with any transition dynamics on some fixed set of states and actions, then there exists an algorithm that reconstructs the agent's reward function to the fullest extent theoretically possible, and that requires only a small (logarithmic) number of experiments.
Action-Conditional Video Prediction using Deep Networks in Atari Games
Motivated by vision-based reinforcement learning (RL) problems, in particular Atari games from the recent benchmark Aracade Learning Environment (ALE), we consider spatio-temporal prediction problems where future (image-)frames are dependent on control variables or actions as well as previous frames.
Deep Learning for Real-Time Atari Game Play Using Offline Monte-Carlo Tree Search Planning
The combination of modern Reinforcement Learning and Deep Learning approaches holds the promise of making significant progress on challenging applications requiring both rich perception and policy-selection.
Learning to Make Predictions In Partially Observable Environments Without a Generative Model
We formalize the problem of learning a prediction profile model as a transformation of the original model-learning problem, and show empirically that one can learn prediction profile models that make a small set of important predictions even in systems that are too complex for standard generative models.
Reward Mapping for Transfer in Long-Lived Agents
We demonstrate that our approach can substantially improve the agent's performance relative to other approaches, including an approach that transfers policies.
Graphical Models for Game Theory
The interpretation is that the payoff to player i is determined entirely by the actions of player i and his neighbors in the graph, and thus the payoff matrix to player i is indexed only by these players.
Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning
In particular, we show that options may be used interchangeably with primitive actions in planning methods such as dynamic programming and in learning methods such as Q-learning.
