Search Results for author:
Found 36 papers, 14 papers with code
Compound Returns Reduce Variance in Reinforcement Learning
Multistep returns, such as $n$-step returns and $\lambda$-returns, are commonly used to improve the sample efficiency of reinforcement learning (RL) methods.
GVFs in the Real World: Making Predictions Online for Water Treatment
In this paper we investigate the use of reinforcement-learning based prediction approaches for a real drinking-water treatment plant.
Harnessing Discrete Representations For Continual Reinforcement Learning
In this work, we provide a thorough empirical investigation of the advantages of representing observations as vectors of categorical values within the context of reinforcement learning.
Directions of Curvature as an Explanation for Loss of Plasticity
Loss of plasticity is a phenomenon in which neural networks lose their ability to learn from new experience.
Recurrent Linear Transformers
In this paper we introduce recurrent alternatives to the transformer self-attention mechanism that offer a context-independent inference cost, leverage long-range dependencies effectively, and perform well in practice.
Proper Laplacian Representation Learning
The ability to learn good representations of states is essential for solving large reinforcement learning problems, where exploration, generalization, and transfer are particularly challenging.
Loss of Plasticity in Continual Deep Reinforcement Learning
The ability to learn continually is essential in a complex and changing world.
Trajectory-Aware Eligibility Traces for Off-Policy Reinforcement Learning
Off-policy learning from multistep returns is crucial for sample-efficient reinforcement learning, but counteracting off-policy bias without exacerbating variance is challenging.
Deep Laplacian-based Options for Temporally-Extended Exploration
In this paper we address these limitations and show how recent results for directly approximating the eigenfunctions of the Laplacian can be leveraged to truly scale up options-based exploration.
Agent-State Construction with Auxiliary Inputs
Finally, we show that this approach is complementary to state-of-the-art methods such as recurrent neural networks and truncated back-propagation through time, and acts as a heuristic that facilitates longer temporal credit assignment, leading to better performance.
Investigating the Properties of Neural Network Representations in Reinforcement Learning
In this paper we investigate the properties of representations learned by deep reinforcement learning systems.
Temporal Abstractions-Augmented Temporally Contrastive Learning: An Alternative to the Laplacian in RL
In reinforcement learning, the graph Laplacian has proved to be a valuable tool in the task-agnostic setting, with applications ranging from skill discovery to reward shaping.
Reward-Respecting Subtasks for Model-Based Reinforcement Learning
Each subtask is solved to produce an option, and then a model of the option is learned and made available to the planning process.
Model-based Reinforcement Learning
reinforcement-learning
+1
Temporal Abstraction in Reinforcement Learning with the Successor Representation
In this paper, we argue that the successor representation (SR), which encodes states based on the pattern of state visitation that follows them, can be seen as a natural substrate for the discovery and use of temporal abstractions.
On Bonus-Based Exploration Methods in the Arcade Learning Environment
Research on exploration in reinforcement learning, as applied to Atari 2600 game-playing, has emphasized tackling difficult exploration problems such as Montezuma's Revenge (Bellemare et al., 2016).
A general class of surrogate functions for stable and efficient reinforcement learning
Common policy gradient methods rely on the maximization of a sequence of surrogate functions.
Exploration-Driven Representation Learning in Reinforcement Learning
In this work, we introduce a method that explicitly couples representation learning with exploration when the agent is not provided with a uniform prior over the state space.
Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learning
Specifically, we introduce a theoretically motivated policy similarity metric (PSM) for measuring behavioral similarity between states.
Beyond variance reduction: Understanding the true impact of baselines on policy optimization
Traditionally, stochastic optimization theory predicts that learning dynamics are governed by the curvature of the loss function and the noise of the gradient estimates.
An operator view of policy gradient methods
We cast policy gradient methods as the repeated application of two operators: a policy improvement operator $\mathcal{I}$, which maps any policy $\pi$ to a better one $\mathcal{I}\pi$, and a projection operator $\mathcal{P}$, which finds the best approximation of $\mathcal{I}\pi$ in the set of realizable policies.
Exploration in Reinforcement Learning with Deep Covering Options
While many option discovery methods have been proposed to accelerate exploration in reinforcement learning, they are often heuristic.
On Bonus Based Exploration Methods In The Arcade Learning Environment
Research on exploration in reinforcement learning, as applied to Atari 2600 game-playing, has emphasized tackling difficult exploration problems such as Montezuma's Revenge (Bellemare et al., 2016).
Benchmarking Bonus-Based Exploration Methods on the Arcade Learning Environment
This paper provides an empirical evaluation of recently developed exploration algorithms within the Arcade Learning Environment (ALE).
Generalization and Regularization in DQN
Deep reinforcement learning algorithms have shown an impressive ability to learn complex control policies in high-dimensional tasks.
Count-Based Exploration with the Successor Representation
In this paper we introduce a simple approach for exploration in reinforcement learning (RL) that allows us to develop theoretically justified algorithms in the tabular case but that is also extendable to settings where function approximation is required.
Ranked #16 on
Atari Games
on Atari 2600 Venture
Accelerating Learning in Constructive Predictive Frameworks with the Successor Representation
As a primary contribution of this work, we show that using SR-based predictions can improve sample efficiency and learning speed in a continual learning setting where new predictions are incrementally added and learned over time.
The Eigenoption-Critic Framework
Eigenoptions (EOs) have been recently introduced as a promising idea for generating a diverse set of options through the graph Laplacian, having been shown to allow efficient exploration.
Efficient Exploration
Hierarchical Reinforcement Learning
+1
Eigenoption Discovery through the Deep Successor Representation
Options in reinforcement learning allow agents to hierarchically decompose a task into subtasks, having the potential to speed up learning and planning.
Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agents
The Arcade Learning Environment (ALE) is an evaluation platform that poses the challenge of building AI agents with general competency across dozens of Atari 2600 games.
A Laplacian Framework for Option Discovery in Reinforcement Learning
Representation learning and option discovery are two of the biggest challenges in reinforcement learning (RL).
Introspective Agents: Confidence Measures for General Value Functions
Agents of general intelligence deployed in real-world scenarios must adapt to ever-changing environmental conditions.
Learning Purposeful Behaviour in the Absence of Rewards
In the reinforcement learning framework, goals are encoded as reward functions that guide agent behaviour, and the sum of observed rewards provide a notion of progress.
True Online Temporal-Difference Learning
Our results suggest that the true online methods indeed dominate the regular methods.
State of the Art Control of Atari Games Using Shallow Reinforcement Learning
The recently introduced Deep Q-Networks (DQN) algorithm has gained attention as one of the first successful combinations of deep neural networks and reinforcement learning.
Domain-Independent Optimistic Initialization for Reinforcement Learning
In Reinforcement Learning (RL), it is common to use optimistic initialization of value functions to encourage exploration.
A Methodology for Player Modeling based on Machine Learning
We also presented a generic approach to deal with player modeling using ML, and we instantiated this approach to model players' preferences in the game Civilization IV.
