Search Results for author:
Found 7 papers, 4 papers with code
A Theory of Unimodal Bias in Multimodal Learning
A long unimodal phase can lead to a generalization deficit and permanent unimodal bias in the overparametrized regime.
Actionable Neural Representations: Grid Cells from Minimal Constraints
We suggest the brain must represent this consistent meaning of actions across space, as it allows you to find new short-cuts and navigate in unfamiliar settings.
Powerpropagation: A sparsity inducing weight reparameterisation
The training of sparse neural networks is becoming an increasingly important tool for reducing the computational footprint of models at training and evaluation, as well enabling the effective scaling up of models.
Towards Biologically Plausible Convolutional Networks
This requires the network to pause occasionally for a sleep-like phase of "weight sharing".
Kernelized information bottleneck leads to biologically plausible 3-factor Hebbian learning in deep networks
The state-of-the art machine learning approach to training deep neural networks, backpropagation, is implausible for real neural networks: neurons need to know their outgoing weights; training alternates between a bottom-up forward pass (computation) and a top-down backward pass (learning); and the algorithm often needs precise labels of many data points.
Synaptic plasticity as Bayesian inference
They then use that uncertainty to adjust their learning rates, with more uncertain weights having higher learning rates.
How biased are maximum entropy models?
However, maximum entropy models fit to small data sets can be subject to sampling bias; i. e. the true entropy of the data can be severely underestimated.
