Search Results for author:
Found 82 papers, 21 papers with code
Population-aware Online Mirror Descent for Mean-Field Games by Deep Reinforcement Learning
Mean Field Games (MFGs) have the ability to handle large-scale multi-agent systems, but learning Nash equilibria in MFGs remains a challenging task.
MusicRL: Aligning Music Generation to Human Preferences
MusicRL is a pretrained autoregressive MusicLM (Agostinelli et al., 2023) model of discrete audio tokens finetuned with reinforcement learning to maximise sequence-level rewards.
Closing the Gap between TD Learning and Supervised Learning -- A Generalisation Point of View
Based on this analysis, we construct new datasets to explicitly test for this property, revealing that SL-based methods lack this stitching property and hence fail to perform combinatorial generalization.
Gemini: A Family of Highly Capable Multimodal Models
This report introduces a new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding.
Ranked #1 on
Multi-task Language Understanding
on MMLU
(using extra training data)
Learning Discrete-Time Major-Minor Mean Field Games
We propose a novel discrete time version of major-minor MFGs (M3FGs), along with a learning algorithm based on fictitious play and partitioning the probability simplex.
A Survey of Temporal Credit Assignment in Deep Reinforcement Learning
In this survey, we review the state of the art of Temporal Credit Assignment (CA) in deep RL.
DRIFT: Deep Reinforcement Learning for Intelligent Floating Platforms Trajectories
This investigation introduces a novel deep reinforcement learning-based suite to control floating platforms in both simulated and real-world environments.
Offline Reinforcement Learning with On-Policy Q-Function Regularization
In this work, we propose to regularize towards the Q-function of the behavior policy instead of the behavior policy itself, under the premise that the Q-function can be estimated more reliably and easily by a SARSA-style estimate and handles the extrapolation error more straightforwardly.
A Connection between One-Step Regularization and Critic Regularization in Reinforcement Learning
One-step methods perform regularization by doing just a single step of policy improvement, while critic regularization methods do many steps of policy improvement with a regularized objective.
On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes
Instead of solely relying on a fixed set of output sequences, GKD trains the student on its self-generated output sequences by leveraging feedback from the teacher on such sequences.
Factually Consistent Summarization via Reinforcement Learning with Textual Entailment Feedback
Despite the seeming success of contemporary grounded text generation systems, they often tend to generate factually inconsistent text with respect to their input.
Abstractive Text Summarization
Natural Language Inference
+2
The Curious Price of Distributional Robustness in Reinforcement Learning with a Generative Model
Assuming access to a generative model that draws samples based on the nominal MDP, we characterize the sample complexity of RMDPs when the uncertainty set is specified via either the total variation (TV) distance or $\chi^2$ divergence.
Regularization and Variance-Weighted Regression Achieves Minimax Optimality in Linear MDPs: Theory and Practice
Mirror descent value iteration (MDVI), an abstraction of Kullback-Leibler (KL) and entropy-regularized reinforcement learning (RL), has served as the basis for recent high-performing practical RL algorithms.
Get Back Here: Robust Imitation by Return-to-Distribution Planning
We consider the Imitation Learning (IL) setup where expert data are not collected on the actual deployment environment but on a different version.
Twice Regularized Markov Decision Processes: The Equivalence between Robustness and Regularization
We then generalize regularized MDPs to twice regularized MDPs ($\text{R}^2$ MDPs), i. e., MDPs with $\textit{both}$ value and policy regularization.
Towards Minimax Optimality of Model-based Robust Reinforcement Learning
In this paper, we consider uncertainty sets defined with an $L_p$-ball (recovering the TV case), and study the sample complexity of \emph{any} planning algorithm (with high accuracy guarantee on the solution) applied to an empirical RMDP estimated using the generative model.
Policy Gradient for Rectangular Robust Markov Decision Processes
We provide a closed-form expression for the worst occupation measure.
Extreme Q-Learning: MaxEnt RL without Entropy
Using EVT, we derive our \emph{Extreme Q-Learning} framework and consequently online and, for the first time, offline MaxEnt Q-learning algorithms, that do not explicitly require access to a policy or its entropy.
Policy Mirror Ascent for Efficient and Independent Learning in Mean Field Games
Instead, we show that $N$ agents running policy mirror ascent converge to the Nash equilibrium of the regularized game within $\widetilde{\mathcal{O}}(\varepsilon^{-2})$ samples from a single sample trajectory without a population generative model, up to a standard $\mathcal{O}(\frac{1}{\sqrt{N}})$ error due to the mean field.
On the importance of data collection for training general goal-reaching policies
Are the quantity and quality of data truly transformative to the performance of a general controller?
Learning Correlated Equilibria in Mean-Field Games
The designs of many large-scale systems today, from traffic routing environments to smart grids, rely on game-theoretic equilibrium concepts.
KL-Entropy-Regularized RL with a Generative Model is Minimax Optimal
In this work, we consider and analyze the sample complexity of model-free reinforcement learning with a generative model.
Learning in Mean Field Games: A Survey
Non-cooperative and cooperative games with a very large number of players have many applications but remain generally intractable when the number of players increases.
Learning Energy Networks with Generalized Fenchel-Young Losses
To learn the parameters of the energy function, the solution to that optimization problem is typically fed into a loss function.
Scalable Deep Reinforcement Learning Algorithms for Mean Field Games
One limiting factor to further scale up using RL is that existing algorithms to solve MFGs require the mixing of approximated quantities such as strategies or $q$-values.
Lazy-MDPs: Towards Interpretable Reinforcement Learning by Learning When to Act
We deem those states and corresponding actions important since they explain the difference in performance between the default and the new, lazy policy.
Continuous Control with Action Quantization from Demonstrations
The proposed approach consists in learning a discretization of continuous action spaces from human demonstrations.
Twice regularized MDPs and the equivalence between robustness and regularization
We finally generalize regularized MDPs to twice regularized MDPs (R${}^2$ MDPs), i. e., MDPs with $\textit{both}$ value and policy regularization.
Large Batch Experience Replay
The optimal sampling distribution being intractable, we make several approximations providing good results in practice and introduce, among others, LaBER (Large Batch Experience Replay), an easy-to-code and efficient method for sampling the replay buffer.
Generalization in Mean Field Games by Learning Master Policies
Mean Field Games (MFGs) can potentially scale multi-agent systems to extremely large populations of agents.
A general class of surrogate functions for stable and efficient reinforcement learning
Common policy gradient methods rely on the maximization of a sequence of surrogate functions.
IQ-Learn: Inverse soft-Q Learning for Imitation
In many sequential decision-making problems (e. g., robotics control, game playing, sequential prediction), human or expert data is available containing useful information about the task.
Ranked #1 on
MuJoCo Games
on Walker2d
Offline Reinforcement Learning as Anti-Exploration
This is the converse of exploration in RL, which favors such actions.
There Is No Turning Back: A Self-Supervised Approach for Reversibility-Aware Reinforcement Learning
We propose to learn to distinguish reversible from irreversible actions for better informed decision-making in Reinforcement Learning (RL).
Concave Utility Reinforcement Learning: the Mean-Field Game Viewpoint
Mean-field Games (MFGs) are a continuous approximation of many-agent RL.
What Matters for Adversarial Imitation Learning?
To tackle this issue, we implement more than 50 of these choices in a generic adversarial imitation learning framework and investigate their impacts in a large-scale study (>500k trained agents) with both synthetic and human-generated demonstrations.
Hyperparameter Selection for Imitation Learning
The vast literature in imitation learning mostly considers this reward function to be available for HP selection, but this is not a realistic setting.
Mean Field Games Flock! The Reinforcement Learning Way
We present a method enabling a large number of agents to learn how to flock, which is a natural behavior observed in large populations of animals.
Offline Reinforcement Learning with Pseudometric Learning
In the presence of function approximation, and under the assumption of limited coverage of the state-action space of the environment, it is necessary to enforce the policy to visit state-action pairs close to the support of logged transitions.
Scaling up Mean Field Games with Online Mirror Descent
We address scaling up equilibrium computation in Mean Field Games (MFGs) using Online Mirror Descent (OMD).
How To Train Your HERON
In this paper we apply Deep Reinforcement Learning (Deep RL) and Domain Randomization to solve a navigation task in a natural environment relying solely on a 2D laser scanner.
Adversarially Guided Actor-Critic
Despite definite success in deep reinforcement learning problems, actor-critic algorithms are still confronted with sample inefficiency in complex environments, particularly in tasks where efficient exploration is a bottleneck.
What Matters for On-Policy Deep Actor-Critic Methods? A Large-Scale Study
In recent years, reinforcement learning (RL) has been successfully applied to many different continuous control tasks.
Self-Imitation Advantage Learning
Self-imitation learning is a Reinforcement Learning (RL) method that encourages actions whose returns were higher than expected, which helps in hard exploration and sparse reward problems.
Leverage the Average: an Analysis of KL Regularization in Reinforcement Learning
Recent Reinforcement Learning (RL) algorithms making use of Kullback-Leibler (KL) regularization as a core component have shown outstanding performance.
Munchausen Reinforcement Learning
Bootstrapping is a core mechanism in Reinforcement Learning (RL).
Ranked #9 on
Atari Games
on Atari-57
Fictitious Play for Mean Field Games: Continuous Time Analysis and Applications
In this paper, we deepen the analysis of continuous time Fictitious Play learning algorithm to the consideration of various finite state Mean Field Game settings (finite horizon, $\gamma$-discounted), allowing in particular for the introduction of an additional common noise.
Show me the Way: Intrinsic Motivation from Demonstrations
Using an inverse RL approach, we show that complex exploration behaviors, reflecting different motivations, can be learnt and efficiently used by RL agents to solve tasks for which exhaustive exploration is prohibitive.
What Matters In On-Policy Reinforcement Learning? A Large-Scale Empirical Study
In recent years, on-policy reinforcement learning (RL) has been successfully applied to many different continuous control tasks.
Primal Wasserstein Imitation Learning
Imitation Learning (IL) methods seek to match the behavior of an agent with that of an expert.
Stable and Efficient Policy Evaluation
Policy evaluation algorithms are essential to reinforcement learning due to their ability to predict the performance of a policy.
Leverage the Average: an Analysis of KL Regularization in RL
Recent Reinforcement Learning (RL) algorithms making use of Kullback-Leibler (KL) regularization as a core component have shown outstanding performance.
Image-Based Place Recognition on Bucolic Environment Across Seasons From Semantic Edge Description
It is built from the wavelet transforms of the image semantic edges.
Momentum in Reinforcement Learning
We adapt the optimization's concept of momentum to reinforcement learning.
On Connections between Constrained Optimization and Reinforcement Learning
In this paper, we draw connections between DP and (constrained) convex optimization.
Self-Attentional Credit Assignment for Transfer in Reinforcement Learning
The ability to transfer knowledge to novel environments and tasks is a sensible desiderata for general learning agents.
ELF: Embedded Localisation of Features in pre-trained CNN
These results show that a CNN trained on a standard task embeds feature location information that is as relevant as when the CNN is specifically trained for feature detection.
On the Convergence of Model Free Learning in Mean Field Games
In order to design scalable algorithms for systems with a large population of interacting agents (e. g. swarms), this paper focuses on Mean Field MAS, where the number of agents is asymptotically infinite.
Modified Actor-Critics
These algorithms iterate policy evaluation followed by a softened policy improvement step.
MULEX: Disentangling Exploitation from Exploration in Deep RL
An agent learning through interactions should balance its action selection process between probing the environment to discover new rewards and using the information acquired in the past to adopt useful behaviour.
Foolproof Cooperative Learning
We prove that in repeated symmetric games, this algorithm is a learning equilibrium.
Deep Conservative Policy Iteration
Conservative Policy Iteration (CPI) is a founding algorithm of Approximate Dynamic Programming (ADP).
CopyCAT: Taking Control of Neural Policies with Constant Attacks
In this setting, the adversary cannot directly modify the agent's state -- its representation of the environment -- but can only attack the agent's observation -- its perception of the environment.
A Theory of Regularized Markov Decision Processes
Many recent successful (deep) reinforcement learning algorithms make use of regularization, generally based on entropy or Kullback-Leibler divergence.
Image-based Natural Language Understanding Using 2D Convolutional Neural Networks
We propose a new approach to natural language understanding in which we consider the input text as an image and apply 2D Convolutional Neural Networks to learn the local and global semantics of the sentences from the variations ofthe visual patterns of words.
Anderson Acceleration for Reinforcement Learning
Anderson acceleration is an old and simple method for accelerating the computation of a fixed point.
Semi-Supervised Domain Adaptation with Representation Learning for Semantic Segmentation across Time
Deep learning generates state-of-the-art semantic segmentation provided that a large number of images together with pixel-wise annotations are available.
Human Activity Recognition using Recurrent Neural Networks
Human activity recognition using smart home sensors is one of the bases of ubiquitous computing in smart environments and a topic undergoing intense research in the field of ambient assisted living.
A Deep Learning Approach for Privacy Preservation in Assisted Living
In the era of Internet of Things (IoT) technologies the potential for privacy invasion is becoming a major concern especially in regards to healthcare data and Ambient Assisted Living (AAL) environments.
Reconstruct & Crush Network
This article introduces an energy-based model that is adversarial regarding data: it minimizes the energy for a given data distribution (the positive samples) while maximizing the energy for another given data distribution (the negative or unlabeled samples).
Is the Bellman residual a bad proxy?
This paper aims at theoretically and empirically comparing two standard optimization criteria for Reinforcement Learning: i) maximization of the mean value and ii) minimization of the Bellman residual.
Difference of Convex Functions Programming Applied to Control with Expert Data
This paper reports applications of Difference of Convex functions (DC) programming to Learning from Demonstrations (LfD) and Reinforcement Learning (RL) with expert data.
Difference of Convex Functions Programming for Reinforcement Learning
Controlling this residual allows controlling the distance to the optimal action-value function, and we show that minimizing an empirical norm of the OBR is consistant in the Vapnik sense.
Kalman Temporal Differences
Because reinforcement learning suffers from a lack of scalability, online value (and Q-) function approximation has received increasing interest this last decade.
Policy Search: Any Local Optimum Enjoys a Global Performance Guarantee
Local Policy Search is a popular reinforcement learning approach for handling large state spaces.
Off-policy Learning with Eligibility Traces: A Survey
In the framework of Markov Decision Processes, off-policy learning, that is the problem of learning a linear approximation of the value function of some fixed policy from one trajectory possibly generated by some other policy.
Inverse Reinforcement Learning through Structured Classification
This paper adresses the inverse reinforcement learning (IRL) problem, that is inferring a reward for which a demonstrated expert behavior is optimal.
Approximate Modified Policy Iteration
Modified policy iteration (MPI) is a dynamic programming (DP) algorithm that contains the two celebrated policy and value iteration methods.
