Search Results for author:
Found 54 papers, 34 papers with code
Scaling Instructable Agents Across Many Simulated Worlds
Building embodied AI systems that can follow arbitrary language instructions in any 3D environment is a key challenge for creating general AI.
Genie: Generative Interactive Environments
We introduce Genie, the first generative interactive environment trained in an unsupervised manner from unlabelled Internet videos.
Managing AI Risks in an Era of Rapid Progress
In this short consensus paper, we outline risks from upcoming, advanced AI systems.
Quality-Diversity through AI Feedback
In many text-generation problems, users may prefer not only a single response, but a diverse range of high-quality outputs from which to choose.
Quality Diversity through Human Feedback
Meanwhile, Quality Diversity (QD) algorithms excel at identifying diverse and high-quality solutions but often rely on manually crafted diversity metrics.
Reset It and Forget It: Relearning Last-Layer Weights Improves Continual and Transfer Learning
We show that our zapping procedure results in improved transfer accuracy and/or more rapid adaptation in both standard fine-tuning and continual learning settings, while being simple to implement and computationally efficient.
First-Explore, then Exploit: Meta-Learning Intelligent Exploration
We argue a core barrier prohibiting many RL approaches from learning intelligent exploration is that the methods attempt to explore and exploit simultaneously, which harms both exploration and exploitation as the goals often conflict.
OMNI: Open-endedness via Models of human Notions of Interestingness
An Achilles Heel of open-endedness research is the inability to quantify (and thus prioritize) tasks that are not just learnable, but also $\textit{interesting}$ (e. g., worthwhile and novel).
Thought Cloning: Learning to Think while Acting by Imitating Human Thinking
We hypothesize one reason for such cognitive deficiencies is that they lack the benefits of thinking in language and that we can improve AI agents by training them to think like humans do.
Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videos
Pretraining on noisy, internet-scale datasets has been heavily studied as a technique for training models with broad, general capabilities for text, images, and other modalities.
Continual learning under domain transfer with sparse synaptic bursting
Past efforts to engineer such systems have sought to build or regulate artificial neural networks using disjoint sets of weights that are uniquely sensitive to specific tasks or inputs.
Multi-task curriculum learning in a complex, visual, hard-exploration domain: Minecraft
An important challenge in reinforcement learning is training agents that can solve a wide variety of tasks.
Open Questions in Creating Safe Open-ended AI: Tensions Between Control and Creativity
This paper proposes that open-ended evolution and artificial life have much to contribute towards the understanding of open-ended AI, focusing here in particular on the safety of open-ended search.
Synthetic Petri Dish: A Novel Surrogate Model for Rapid Architecture Search
Neural Architecture Search (NAS) explores a large space of architectural motifs -- a compute-intensive process that often involves ground-truth evaluation of each motif by instantiating it within a large network, and training and evaluating the network with thousands of domain-specific data samples.
First return, then explore
The promise of reinforcement learning is to solve complex sequential decision problems autonomously by specifying a high-level reward function only.
Ranked #1 on
Atari Games
on Atari 2600 Montezuma's Revenge
Fiber: A Platform for Efficient Development and Distributed Training for Reinforcement Learning and Population-Based Methods
Recent advances in machine learning are consistently enabled by increasing amounts of computation.
Enhanced POET: Open-Ended Reinforcement Learning through Unbounded Invention of Learning Challenges and their Solutions
Creating open-ended algorithms, which generate their own never-ending stream of novel and appropriately challenging learning opportunities, could help to automate and accelerate progress in machine learning.
Scaling MAP-Elites to Deep Neuroevolution
Quality-Diversity (QD) algorithms, and MAP-Elites (ME) in particular, have proven very useful for a broad range of applications including enabling real robots to recover quickly from joint damage, solving strongly deceptive maze tasks or evolving robot morphologies to discover new gaits.
Backpropamine: training self-modifying neural networks with differentiable neuromodulated plasticity
We show that neuromodulated plasticity improves the performance of neural networks on both reinforcement learning and supervised learning tasks.
Learning to Continually Learn
Continual lifelong learning requires an agent or model to learn many sequentially ordered tasks, building on previous knowledge without catastrophically forgetting it.
Generative Teaching Networks: Accelerating Neural Architecture Search by Learning to Generate Synthetic Training Data
This paper introduces GTNs, discusses their potential, and showcases that they can substantially accelerate learning.
A deep active learning system for species identification and counting in camera trap images
However, the accuracy of results depends on the amount, quality, and diversity of the data available to train models, and the literature has focused on projects with millions of relevant, labeled training images.
Evolvability ES: Scalable and Direct Optimization of Evolvability
Designing evolutionary algorithms capable of uncovering highly evolvable representations is an open challenge; such evolvability is important because it accelerates evolution and enables fast adaptation to changing circumstances.
AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence
Because it it may be the fastest path to general AI and because it is inherently scientifically interesting to understand the conditions in which a simple algorithm can produce general AI (as happened on Earth where Darwinian evolution produced human intelligence), I argue that the pursuit of AI-GAs should be considered a new grand challenge of computer science research.
Understanding Neural Networks via Feature Visualization: A survey
A neuroscience method to understanding the brain is to find and study the preferred stimuli that highly activate an individual cell or groups of cells.
Go-Explore: a New Approach for Hard-Exploration Problems
Go-Explore can also harness human-provided domain knowledge and, when augmented with it, scores a mean of over 650k points on Montezuma's Revenge.
Ranked #1 on
Atari Games
on Atari 2600 Pitfall!
Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their Solutions
Our results show that POET produces a diverse range of sophisticated behaviors that solve a wide range of environmental challenges, many of which cannot be solved by direct optimization alone, or even through a direct-path curriculum-building control algorithm introduced to highlight the critical role of open-endedness in solving ambitious challenges.
Robustness to Out-of-Distribution Inputs via Task-Aware Generative Uncertainty
Our method estimates an uncertainty measure about the model's prediction, taking into account an explicit (generative) model of the observation distribution to handle out-of-distribution inputs.
An Atari Model Zoo for Analyzing, Visualizing, and Comparing Deep Reinforcement Learning Agents
We lessen this friction, by (1) training several algorithms at scale and releasing trained models, (2) integrating with a previous Deep RL model release, and (3) releasing code that makes it easy for anyone to load, visualize, and analyze such models.
Evolving Multimodal Robot Behavior via Many Stepping Stones with the Combinatorial Multi-Objective Evolutionary Algorithm
Lastly, we show that, in contrast to NSGA-II and Lexicase Selection, CMOEA can effectively leverage secondary objectives to achieve state-of-the-art results on the robotics task.
Deep Curiosity Search: Intra-Life Exploration Can Improve Performance on Challenging Deep Reinforcement Learning Problems
The strong performance of DeepCS on these sparse- and dense-reward tasks suggests that encouraging intra-life novelty is an interesting, new approach for improving performance in Deep RL and motivates further research into hybridizing across-training and intra-life exploration methods.
VINE: An Open Source Interactive Data Visualization Tool for Neuroevolution
Recent advances in deep neuroevolution have demonstrated that evolutionary algorithms, such as evolution strategies (ES) and genetic algorithms (GA), can scale to train deep neural networks to solve difficult reinforcement learning (RL) problems.
Differentiable plasticity: training plastic neural networks with backpropagation
How can we build agents that keep learning from experience, quickly and efficiently, after their initial training?
The Surprising Creativity of Digital Evolution: A Collection of Anecdotes from the Evolutionary Computation and Artificial Life Research Communities
Biological evolution provides a creative fount of complex and subtle adaptations, often surprising the scientists who discover them.
Improving Exploration in Evolution Strategies for Deep Reinforcement Learning via a Population of Novelty-Seeking Agents
Evolution strategies (ES) are a family of black-box optimization algorithms able to train deep neural networks roughly as well as Q-learning and policy gradient methods on challenging deep reinforcement learning (RL) problems, but are much faster (e. g. hours vs. days) because they parallelize better.
On the Relationship Between the OpenAI Evolution Strategy and Stochastic Gradient Descent
Because stochastic gradient descent (SGD) has shown promise optimizing neural networks with millions of parameters and few if any alternatives are known to exist, it has moved to the heart of leading approaches to reinforcement learning (RL).
Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning
Here we demonstrate they can: we evolve the weights of a DNN with a simple, gradient-free, population-based genetic algorithm (GA) and it performs well on hard deep RL problems, including Atari and humanoid locomotion.
ES Is More Than Just a Traditional Finite-Difference Approximator
However, this ES optimizes for a different gradient than just reward: It optimizes for the average reward of the entire population, thereby seeking parameters that are robust to perturbation.
Safe Mutations for Deep and Recurrent Neural Networks through Output Gradients
While neuroevolution (evolving neural networks) has a successful track record across a variety of domains from reinforcement learning to artificial life, it is rarely applied to large, deep neural networks.
Diffusion-based neuromodulation can eliminate catastrophic forgetting in simple neural networks
On the simple diagnostic problem from the prior work, diffusion-based neuromodulation 1) induces task-specific learning in groups of nodes and connections (task-specific localized learning), which 2) produces functional modules for each subtask, and 3) yields higher performance by eliminating catastrophic forgetting.
The Emergence of Canalization and Evolvability in an Open-Ended, Interactive Evolutionary System
In this paper we reveal a unique system in which canalization did emerge in computational evolution.
Automatically identifying, counting, and describing wild animals in camera-trap images with deep learning
Having accurate, detailed, and up-to-date information about the location and behavior of animals in the wild would revolutionize our ability to study and conserve ecosystems.
Plug & Play Generative Networks: Conditional Iterative Generation of Images in Latent Space
PPGNs are composed of 1) a generator network G that is capable of drawing a wide range of image types and 2) a replaceable "condition" network C that tells the generator what to draw.
Synthesizing the preferred inputs for neurons in neural networks via deep generator networks
Understanding the inner workings of such computational brains is both fascinating basic science that is interesting in its own right - similar to why we study the human brain - and will enable researchers to further improve DNNs.
Multifaceted Feature Visualization: Uncovering the Different Types of Features Learned By Each Neuron in Deep Neural Networks
Here, we introduce an algorithm that explicitly uncovers the multiple facets of each neuron by producing a synthetic visualization of each of the types of images that activate a neuron.
Convergent Learning: Do different neural networks learn the same representations?
Recent success in training deep neural networks have prompted active investigation into the features learned on their intermediate layers.
Understanding Neural Networks Through Deep Visualization
The first is a tool that visualizes the activations produced on each layer of a trained convnet as it processes an image or video (e. g. a live webcam stream).
The evolutionary origins of hierarchy
Hierarchical organization -- the recursive composition of sub-modules -- is ubiquitous in biological networks, including neural, metabolic, ecological, and genetic regulatory networks, and in human-made systems, such as large organizations and the Internet.
Illuminating search spaces by mapping elites
Interestingly, because MAP-Elites explores more of the search space, it also tends to find a better overall solution than state-of-the-art search algorithms.
Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images
Here we show a related result: it is easy to produce images that are completely unrecognizable to humans, but that state-of-the-art DNNs believe to be recognizable objects with 99. 99% confidence (e. g. labeling with certainty that white noise static is a lion).
How transferable are features in deep neural networks?
Such first-layer features appear not to be specific to a particular dataset or task, but general in that they are applicable to many datasets and tasks.
Robots that can adapt like animals
As robots leave the controlled environments of factories to autonomously function in more complex, natural environments, they will have to respond to the inevitable fact that they will become damaged.
Hands-free Evolution of 3D-printable Objects via Eye Tracking
Interactive evolution has shown the potential to create amazing and complex forms in both 2-D and 3-D settings.
The evolutionary origins of modularity
A central biological question is how natural organisms are so evolvable (capable of quickly adapting to new environments).
