Search Results for author:
Found 27 papers, 10 papers with code
Is Scaling Learned Optimizers Worth It? Evaluating The Value of VeLO's 4000 TPU Months
We analyze VeLO (versatile learned optimizer), the largest scale attempt to train a general purpose "foundational" optimizer to date.
Evaluating the Evaluators: Are Current Few-Shot Learning Benchmarks Fit for Purpose?
Numerous benchmarks for Few-Shot Learning have been proposed in the last decade.
Meta Omnium: A Benchmark for General-Purpose Learning-to-Learn
Meta-learning and other approaches to few-shot learning are widely studied for image recognition, and are increasingly applied to other vision tasks such as pose estimation and dense prediction.
Effectiveness of Debiasing Techniques: An Indigenous Qualitative Analysis
An indigenous perspective on the effectiveness of debiasing techniques for pre-trained language models (PLMs) is presented in this paper.
Amortised Invariance Learning for Contrastive Self-Supervision
Contrastive self-supervised learning methods famously produce high quality transferable representations by learning invariances to different data augmentations.
Quality Diversity for Visual Pre-Training
Notably, the augmentations used in both supervised and self-supervised training lead to features with high invariance to spatial and appearance transformations.
Attacking Adversarial Defences by Smoothing the Loss Landscape
This paper investigates a family of methods for defending against adversarial attacks that owe part of their success to creating a noisy, discontinuous, or otherwise rugged loss landscape that adversaries find difficult to navigate.
HyperInvariances: Amortizing Invariance Learning
Providing invariances in a given learning task conveys a key inductive bias that can lead to sample-efficient learning and good generalisation, if correctly specified.
Lessons learned from the NeurIPS 2021 MetaDL challenge: Backbone fine-tuning without episodic meta-learning dominates for few-shot learning image classification
Although deep neural networks are capable of achieving performance superior to humans on various tasks, they are notorious for requiring large amounts of data and computing resources, restricting their success to domains where such resources are available.
Meta Mirror Descent: Optimiser Learning for Fast Convergence
The resulting framework, termed Meta Mirror Descent (MetaMD), learns to accelerate optimisation speed.
Finding lost DG: Explaining domain generalization via model complexity
However much of the work in general purpose DG is heuristically motivated, as the DG problem is hard to model formally; and recent evaluations have cast doubt on existing methods' practical efficacy -- in particular compared to a well tuned empirical risk minimisation baseline.
Why Do Self-Supervised Models Transfer? Investigating the Impact of Invariance on Downstream Tasks
We show that learned invariances strongly affect downstream task performance and confirm that different downstream tasks benefit from polar opposite (in)variances, leading to performance loss when the standard augmentation strategy is used.
Self-Supervised Representation Learning: Introduction, Advances and Challenges
Self-supervised representation learning methods aim to provide powerful deep feature learning without the requirement of large annotated datasets, thus alleviating the annotation bottleneck that is one of the main barriers to practical deployment of deep learning today.
Active Altruism Learning and Information Sufficiency for Autonomous Driving
Safe interaction between vehicles requires the ability to choose actions that reveal the preferences of the other vehicles.
Loss Function Learning for Domain Generalization by Implicit Gradient
We take a different approach, and explore the impact of the ERM loss function on out-of-domain generalisation.
Searching for Robustness: Loss Learning for Noisy Classification Tasks
We present a "learning to learn" approach for automatically constructing white-box classification loss functions that are robust to label noise in the training data.
Shallow Bayesian Meta Learning for Real-World Few-Shot Recognition
Current state-of-the-art few-shot learners focus on developing effective training procedures for feature representations, before using simple, e. g. nearest centroid, classifiers.
Ranked #6 on
Few-Shot Image Classification
on Meta-Dataset
cross-domain few-shot learning
Few-Shot Image Classification
How Well Do Self-Supervised Models Transfer?
We evaluate the transfer performance of 13 top self-supervised models on 40 downstream tasks, including many-shot and few-shot recognition, object detection, and dense prediction.
Weight-Covariance Alignment for Adversarially Robust Neural Networks
Stochastic Neural Networks (SNNs) that inject noise into their hidden layers have recently been shown to achieve strong robustness against adversarial attacks.
Don't Wait, Just Weight: Improving Unsupervised Representations by Learning Goal-Driven Instance Weights
We show that by learning Bayesian instance weights for the unlabelled data, we can improve the downstream classification accuracy by prioritising the most useful instances.
Distance-Based Regularisation of Deep Networks for Fine-Tuning
Our bound is highly relevant for fine-tuning, because providing a network with a good initialisation based on transfer learning means that learning can modify the weights less, and hence achieve tighter generalisation.
Optimising Neural Network Architectures for Provable Adversarial Robustness
Existing Lipschitz-based provable defences to adversarial examples only cover the L2 threat model.
Deep clustering with concrete k-means
We address the problem of simultaneously learning a k-means clustering and deep feature representation from unlabelled data, which is of interest due to the potential of deep k-means to outperform traditional two-step feature extraction and shallow-clustering strategies.
Stochastic Gradient Trees
We present an algorithm for learning decision trees using stochastic gradient information as the source of supervision.
MaxGain: Regularisation of Neural Networks by Constraining Activation Magnitudes
Effective regularisation of neural networks is essential to combat overfitting due to the large number of parameters involved.
Regularisation of Neural Networks by Enforcing Lipschitz Continuity
We investigate the effect of explicitly enforcing the Lipschitz continuity of neural networks with respect to their inputs.
Fast Metric Learning For Deep Neural Networks
Similarity metrics are a core component of many information retrieval and machine learning systems.
