Search Results for author:
Found 33 papers, 23 papers with code
Estimating Q(s,s') with Deterministic Dynamics Gradients
In this paper, we introduce a novel form of a value function, $Q(s, s')$, that expresses the utility of transitioning from a state $s$ to a neighboring state $s'$ and then acting optimally thereafter.
RxRx1: A Dataset for Evaluating Experimental Batch Correction Methods
We propose a classification task designed to evaluate the effectiveness of experimental batch correction methods on these images and examine the performance of a number of correction methods on this task.
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
BIG-bench focuses on tasks that are believed to be beyond the capabilities of current language models.
Exploring unfairness in Integrated Gradients based attribution methods
Integrated Gradients has axioms derived from this heritage with the implication of a similar rigorous, intuitive notion of fairness.
Language Models are Few-shot Multilingual Learners
General-purpose language models have demonstrated impressive capabilities, performing on par with state-of-the-art approaches on a range of downstream natural language processing (NLP) tasks and benchmarks when inferring instructions from very few examples.
When does loss-based prioritization fail?
Each example is propagated forward and backward through the network the same amount of times, independent of how much the example contributes to the learning protocol.
Supermasks in Superposition
We present the Supermasks in Superposition (SupSup) model, capable of sequentially learning thousands of tasks without catastrophic forgetting.
Estimating Q(s,s') with Deep Deterministic Dynamics Gradients
In this paper, we introduce a novel form of value function, $Q(s, s')$, that expresses the utility of transitioning from a state $s$ to a neighboring state $s'$ and then acting optimally thereafter.
Plug and Play Language Models: A Simple Approach to Controlled Text Generation
Large transformer-based language models (LMs) trained on huge text corpora have shown unparalleled generation capabilities.
First-Order Preconditioning via Hypergradient Descent
Standard gradient descent methods are susceptible to a range of issues that can impede training, such as high correlations and different scaling in parameter space. These difficulties can be addressed by second-order approaches that apply a pre-conditioning matrix to the gradient to improve convergence.
LCA: Loss Change Allocation for Neural Network Training
We propose a new window into training called Loss Change Allocation (LCA), in which credit for changes to the network loss is conservatively partitioned to the parameters.
Hamiltonian Neural Networks
Even though neural networks enjoy widespread use, they still struggle to learn the basic laws of physics.
Deconstructing Lottery Tickets: Zeros, Signs, and the Supermask
The recent "Lottery Ticket Hypothesis" paper by Frankle & Carbin showed that a simple approach to creating sparse networks (keeping the large weights) results in models that are trainable from scratch, but only when starting from the same initial weights.
Understanding Neural Networks via Feature Visualization: A survey
A neuroscience method to understanding the brain is to find and study the preferred stimuli that highly activate an individual cell or groups of cells.
Faster Neural Networks Straight from JPEG
In this paper we modify \libjpeg to produce DCT coefficients directly, modify a ResNet-50 network to accommodate the differently sized and strided input, and evaluate performance on ImageNet.
Metropolis-Hastings Generative Adversarial Networks
We introduce the Metropolis-Hastings generative adversarial network (MH-GAN), which combines aspects of Markov chain Monte Carlo and GANs.
An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solution
In this paper we show a striking counterexample to this intuition via the seemingly trivial coordinate transform problem, which simply requires learning a mapping between coordinates in (x, y) Cartesian space and one-hot pixel space.
Ranked #866 on
Image Classification
on ImageNet
Measuring the Intrinsic Dimension of Objective Landscapes
A byproduct of this construction is a simple approach for compressing networks, in some cases by more than 100 times.
The Surprising Creativity of Digital Evolution: A Collection of Anecdotes from the Evolutionary Computation and Artificial Life Research Communities
Biological evolution provides a creative fount of complex and subtle adaptations, often surprising the scientists who discover them.
Proceedings of NIPS 2017 Symposium on Interpretable Machine Learning
This is the Proceedings of NIPS 2017 Symposium on Interpretable Machine Learning, held in Long Beach, California, USA on December 7, 2017
SVCCA: Singular Vector Canonical Correlation Analysis for Deep Learning Dynamics and Interpretability
We propose a new technique, Singular Vector Canonical Correlation Analysis (SVCCA), a tool for quickly comparing two representations in a way that is both invariant to affine transform (allowing comparison between different layers and networks) and fast to compute (allowing more comparisons to be calculated than with previous methods).
Plug & Play Generative Networks: Conditional Iterative Generation of Images in Latent Space
PPGNs are composed of 1) a generator network G that is capable of drawing a wide range of image types and 2) a replaceable "condition" network C that tells the generator what to draw.
Synthesizing the preferred inputs for neurons in neural networks via deep generator networks
Understanding the inner workings of such computational brains is both fascinating basic science that is interesting in its own right - similar to why we study the human brain - and will enable researchers to further improve DNNs.
Multifaceted Feature Visualization: Uncovering the Different Types of Features Learned By Each Neuron in Deep Neural Networks
Here, we introduce an algorithm that explicitly uncovers the multiple facets of each neuron by producing a synthetic visualization of each of the types of images that activate a neuron.
Convergent Learning: Do different neural networks learn the same representations?
Recent success in training deep neural networks have prompted active investigation into the features learned on their intermediate layers.
Recombinator Networks: Learning Coarse-to-Fine Feature Aggregation
Deep neural networks with alternating convolutional, max-pooling and decimation layers are widely used in state of the art architectures for computer vision.
Understanding Neural Networks Through Deep Visualization
The first is a tool that visualizes the activations produced on each layer of a trained convnet as it processes an image or video (e. g. a live webcam stream).
Can deep learning help you find the perfect match?
The answer to this question depends on the personal preferences of the one asking it.
GSNs : Generative Stochastic Networks
We introduce a novel training principle for probabilistic models that is an alternative to maximum likelihood.
Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images
Here we show a related result: it is easy to produce images that are completely unrecognizable to humans, but that state-of-the-art DNNs believe to be recognizable objects with 99. 99% confidence (e. g. labeling with certainty that white noise static is a lion).
How transferable are features in deep neural networks?
Such first-layer features appear not to be specific to a particular dataset or task, but general in that they are applicable to many datasets and tasks.
Deep Generative Stochastic Networks Trainable by Backprop
We introduce a novel training principle for probabilistic models that is an alternative to maximum likelihood.
Hands-free Evolution of 3D-printable Objects via Eye Tracking
Interactive evolution has shown the potential to create amazing and complex forms in both 2-D and 3-D settings.
