Search Results for author:
Found 36 papers, 14 papers with code
Maxwell's Demon at Work: Efficient Pruning by Leveraging Saturation of Neurons
When training deep neural networks, the phenomenon of $\textit{dying neurons}$ $\unicode{x2013}$units that become inactive or saturated, output zero during training$\unicode{x2013}$ has traditionally been viewed as undesirable, linked with optimization challenges, and contributing to plasticity loss in continual learning scenarios.
Do Transformer World Models Give Better Policy Gradients?
We integrate such AWMs into a policy gradient framework that underscores the relationship between network architectures and the policy gradient updates they inherently represent.
Bridging State and History Representations: Understanding Self-Predictive RL
These findings culminate in a set of preliminary guidelines for RL practitioners.
Maximum entropy GFlowNets with soft Q-learning
Generative Flow Networks (GFNs) have emerged as a powerful tool for sampling discrete objects from unnormalized distributions, offering a scalable alternative to Markov Chain Monte Carlo (MCMC) methods.
Course Correcting Koopman Representations
Koopman representations aim to learn features of nonlinear dynamical systems (NLDS) which lead to linear dynamics in the latent space.
Motif: Intrinsic Motivation from Artificial Intelligence Feedback
Exploring rich environments and evaluating one's actions without prior knowledge is immensely challenging.
Policy Optimization in a Noisy Neighborhood: On Return Landscapes in Continuous Control
To conclude, we develop a distribution-aware procedure which finds such paths, navigating away from noisy neighborhoods in order to improve the robustness of a policy.
When Do Transformers Shine in RL? Decoupling Memory from Credit Assignment
The Transformer architecture has been very successful to solve problems that involve long-term dependencies, including in the RL domain.
Goal-conditioned GFlowNets for Controllable Multi-Objective Molecular Design
In recent years, in-silico molecular design has received much attention from the machine learning community.
Designing Biological Sequences via Meta-Reinforcement Learning and Bayesian Optimization
The problem can be framed as a global optimization problem where the objective is an expensive black-box function such that we can query large batches restricted with a limitation of a low number of rounds.
The Primacy Bias in Deep Reinforcement Learning
This work identifies a common flaw of deep reinforcement learning (RL) algorithms: a tendency to rely on early interactions and ignore useful evidence encountered later.
Ranked #8 on
Atari Games 100k
on Atari 100k
Continuous-Time Meta-Learning with Forward Mode Differentiation
Drawing inspiration from gradient-based meta-learning methods with infinitely small gradient steps, we introduce Continuous-Time Meta-Learning (COMLN), a meta-learning algorithm where adaptation follows the dynamics of a gradient vector field.
Myriad: a real-world testbed to bridge trajectory optimization and deep learning
We present Myriad, a testbed written in JAX for learning and planning in real-world continuous environments.
Direct Behavior Specification via Constrained Reinforcement Learning
The standard formulation of Reinforcement Learning lacks a practical way of specifying what are admissible and forbidden behaviors.
Neural Algorithmic Reasoners are Implicit Planners
We find that prior approaches either assume that the environment is provided in such a tabular form -- which is highly restrictive -- or infer "local neighbourhoods" of states to run value iteration over -- for which we discover an algorithmic bottleneck effect.
DistProp: A Scalable Approach to Lagrangian Training via Distributional Approximation
We develop a multiple shooting method for learning in deep neural networks based on the Lagrangian perspective on automatic differentiation.
Control-Oriented Model-Based Reinforcement Learning with Implicit Differentiation
The shortcomings of maximum likelihood estimation in the context of model-based reinforcement learning have been highlighted by an increasing number of papers.
Model-based Reinforcement Learning
reinforcement-learning
+1
An Information-Theoretic Perspective on Credit Assignment in Reinforcement Learning
How do we formalize the challenge of credit assignment in reinforcement learning?
XLVIN: eXecuted Latent Value Iteration Nets
Value Iteration Networks (VINs) have emerged as a popular method to incorporate planning algorithms within deep reinforcement learning, enabling performance improvements on tasks requiring long-range reasoning and understanding of environment dynamics.
Graph neural induction of value iteration
Previously, such planning components have been incorporated through a neural network that partially aligns with the computational graph of value iteration.
TDprop: Does Jacobi Preconditioning Help Temporal Difference Learning?
We investigate whether Jacobi preconditioning, accounting for the bootstrap term in temporal difference (TD) learning, can help boost performance of adaptive optimizers.
Policy Evaluation Networks
The core idea of this paper is to flip this convention and estimate the value of many policies, for a single set of states.
Options of Interest: Temporal Abstraction with Interest Functions
Temporal abstraction refers to the ability of an agent to use behaviours of controllers which act for a limited, variable amount of time.
Entropy Regularization with Discounted Future State Distribution in Policy Gradient Methods
In this work, we propose exploration in policy gradient methods based on maximizing entropy of the discounted future state distribution.
All-Action Policy Gradient Methods: A Numerical Integration Approach
While often stated as an instance of the likelihood ratio trick [Rubinstein, 1989], the original policy gradient theorem [Sutton, 1999] involves an integral over the action space.
Understanding the Curse of Horizon in Off-Policy Evaluation via Conditional Importance Sampling
Surprisingly, we find that in finite horizon MDPs there is no strict variance reduction of per-decision importance sampling or stationary importance sampling, comparing with vanilla importance sampling.
The Barbados 2018 List of Open Issues in Continual Learning
We want to make progress toward artificial general intelligence, namely general-purpose agents that autonomously learn how to competently act in complex environments.
Learning Robust Options
We present a Robust Options Policy Iteration (ROPI) algorithm with convergence guarantees, which learns options that are robust to model uncertainty.
Learnings Options End-to-End for Continuous Action Tasks
We present new results on learning temporally extended actions for continuoustasks, using the options framework (Suttonet al.[1999b], Precup [2000]).
Learning with Options that Terminate Off-Policy
Generally, learning with longer options (like learning with multi-step returns) is known to be more efficient.
OptionGAN: Learning Joint Reward-Policy Options using Generative Adversarial Inverse Reinforcement Learning
Inverse reinforcement learning offers a useful paradigm to learn the underlying reward function directly from expert demonstrations.
When Waiting is not an Option : Learning Options with a Deliberation Cost
Recent work has shown that temporally extended actions (options) can be learned fully end-to-end as opposed to being specified in advance.
Convergent Tree Backup and Retrace with Function Approximation
Off-policy learning is key to scaling up reinforcement learning as it allows to learn about a target policy from the experience generated by a different behavior policy.
A Matrix Splitting Perspective on Planning with Options
We show that the Bellman operator underlying the options framework leads to a matrix splitting, an approach traditionally used to speed up convergence of iterative solvers for large linear systems of equations.
The Option-Critic Architecture
Temporal abstraction is key to scaling up learning and planning in reinforcement learning.
Conditional Computation in Neural Networks for faster models
In this paper, we use reinforcement learning as a tool to optimize conditional computation policies.
