Search Results for author:
Found 21 papers, 8 papers with code
Localized random projections challenge benchmarks for bio-plausible deep learning
An appealing alternative to training deep neural networks is to use one or a few hidden layers with fixed random weights or trained with an unsupervised, local learning rule and train a single readout layer with a supervised, local learning rule.
Should Under-parameterized Student Networks Copy or Average Teacher Weights?
Approximating $f^*$ with a neural network with $n< k$ neurons can thus be seen as fitting an under-parameterized "student" network with $n$ neurons to a "teacher" network with $k$ neurons.
Expand-and-Cluster: Parameter Recovery of Neural Networks
Can we identify the parameters of a neural network by probing its input-output mapping?
Emergent rate-based dynamics in duplicate-free populations of spiking neurons
Can the dynamics of Spiking Neural Networks (SNNs) approximate the dynamics of Recurrent Neural Networks (RNNs)?
MLPGradientFlow: going with the flow of multilayer perceptrons (and finding minima fast and accurately)
MLPGradientFlow is a software package to solve numerically the gradient flow differential equation $\dot \theta = -\nabla \mathcal L(\theta; \mathcal D)$, where $\theta$ are the parameters of a multi-layer perceptron, $\mathcal D$ is some data set, and $\nabla \mathcal L$ is the gradient of a loss function.
A taxonomy of surprise definitions
Going beyond this technical analysis, we propose a taxonomy of surprise definitions and classify them into four conceptual categories based on the quantity they measure: (i) 'prediction surprise' measures a mismatch between a prediction and an observation; (ii) 'change-point detection surprise' measures the probability of a change in the environment; (iii) 'confidence-corrected surprise' explicitly accounts for the effect of confidence; and (iv) 'information gain surprise' measures the belief-update upon a new observation.
Kernel Memory Networks: A Unifying Framework for Memory Modeling
We consider the problem of training a neural network to store a set of patterns with maximal noise robustness.
Neural NID Rules
Abstract object properties and their relations are deeply rooted in human common sense, allowing people to predict the dynamics of the world even in situations that are novel but governed by familiar laws of physics.
Common Sense Reasoning
Model-based Reinforcement Learning
+3
Fitting summary statistics of neural data with a differentiable spiking network simulator
Fitting network models to neural activity is an important tool in neuroscience.
Geometry of the Loss Landscape in Overparameterized Neural Networks: Symmetries and Invariances
For a two-layer overparameterized network of width $ r^*+ h =: m $ we explicitly describe the manifold of global minima: it consists of $ T(r^*, m) $ affine subspaces of dimension at least $ h $ that are connected to one another.
Weight-space symmetry in neural network loss landscapes revisited
In a network of $d-1$ hidden layers with $n_k$ neurons in layers $k = 1, \ldots, d$, we construct continuous paths between equivalent global minima that lead through a `permutation point' where the input and output weight vectors of two neurons in the same hidden layer $k$ collide and interchange.
Weight-space symmetry in deep networks gives rise to permutation saddles, connected by equal-loss valleys across the loss landscape
The permutation symmetry of neurons in each layer of a deep neural network gives rise not only to multiple equivalent global minima of the loss function, but also to first-order saddle points located on the path between the global minima.
Learning in Volatile Environments with the Bayes Factor Surprise
Surprise-based learning allows agents to rapidly adapt to non-stationary stochastic environments characterized by sudden changes.
Biologically plausible deep learning -- but how far can we go with shallow networks?
These spiking models achieve > 98. 2% test accuracy on MNIST, which is close to the performance of rate networks with one hidden layer trained with backpropagation.
GaussianProcesses.jl: A Nonparametric Bayes package for the Julia Language
Gaussian processes are a class of flexible nonparametric Bayesian tools that are widely used across the sciences, and in industry, to model complex data sources.
Learning to Generate Music with BachProp
As deep learning advances, algorithms of music composition increase in performance.
Efficient Model-Based Deep Reinforcement Learning with Variational State Tabulation
Modern reinforcement learning algorithms reach super-human performance on many board and video games, but they are sample inefficient, i. e. they typically require significantly more playing experience than humans to reach an equal performance level.
Is prioritized sweeping the better episodic control?
Episodic control has been proposed as a third approach to reinforcement learning, besides model-free and model-based control, by analogy with the three types of human memory.
Towards deep learning with spiking neurons in energy based models with contrastive Hebbian plasticity
In machine learning, error back-propagation in multi-layer neural networks (deep learning) has been impressively successful in supervised and reinforcement learning tasks.
Algorithmic Composition of Melodies with Deep Recurrent Neural Networks
A big challenge in algorithmic composition is to devise a model that is both easily trainable and able to reproduce the long-range temporal dependencies typical of music.
Sequence learning with hidden units in spiking neural networks
We consider a statistical framework in which recurrent networks of spiking neurons learn to generate spatio-temporal spike patterns.
