Search Results for author:
Found 27 papers, 17 papers with code
Are Vision Language Models Texture or Shape Biased and Can We Steer Them?
If text does indeed influence visual biases, this suggests that we may be able to steer visual biases not just through visual input but also through language: a hypothesis that we confirm through extensive experiments.
Neither hype nor gloom do DNNs justice
Neither the hype exemplified in some exaggerated claims about deep neural networks (DNNs), nor the gloom expressed by Bowers et al. do DNNs as models in vision science justice: DNNs rapidly evolve, and today's limitations are often tomorrow's successes.
Getting aligned on representational alignment
Finally, we lay out open problems in representational alignment where progress can benefit all three of these fields.
Intriguing properties of generative classifiers
What is the best paradigm to recognize objects -- discriminative inference (fast but potentially prone to shortcut learning) or using a generative model (slow but potentially more robust)?
Patch n' Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolution
The ubiquitous and demonstrably suboptimal choice of resizing images to a fixed resolution before processing them with computer vision models has not yet been successfully challenged.
Don't trust your eyes: on the (un)reliability of feature visualizations
Today, visualization methods form the foundation of our knowledge about the internal workings of neural networks, as a type of mechanistic interpretability.
Are Deep Neural Networks Adequate Behavioural Models of Human Visual Perception?
Deep neural networks (DNNs) are machine learning algorithms that have revolutionised computer vision due to their remarkable successes in tasks like object classification and segmentation.
Scaling Vision Transformers to 22 Billion Parameters
The scaling of Transformers has driven breakthrough capabilities for language models.
Ranked #1 on
Zero-Shot Transfer Image Classification
on ObjectNet
Beyond neural scaling laws: beating power law scaling via data pruning
Widely observed neural scaling laws, in which error falls off as a power of the training set size, model size, or both, have driven substantial performance improvements in deep learning.
The developmental trajectory of object recognition robustness: children are like small adults but unlike big deep neural networks
Unlike adults', whose object recognition performance is robust against a wide range of image distortions, DNNs trained on standard ImageNet (1. 3M images) perform poorly on distorted images.
Trivial or impossible -- dichotomous data difficulty masks model differences (on ImageNet and beyond)
We find that the ImageNet validation set, among others, suffers from dichotomous data difficulty (DDD): For the range of investigated models and their accuracies, it is dominated by 46. 0% "trivial" and 11. 5% "impossible" images (beyond label errors).
Out-of-distribution robustness: Limited image exposure of a four-year-old is enough to outperform ResNet-50
Recent gains in model robustness towards out-of-distribution images are predominantly achieved through ever-increasing large-scale datasets.
Trivial or Impossible --- dichotomous data difficulty masks model differences (on ImageNet and beyond)
We find that the ImageNet validation set, among others, suffers from dichotomous data difficulty (DDD): For the range of investigated models and their accuracies, it is dominated by 46. 0% ``trivial'' and 11. 5% ``impossible'' images (beyond label errors).
ImageNet suffers from dichotomous data difficulty
We find that the ImageNet validation set suffers from dichotomous data difficulty (DDD): For the range of investigated models and their accuracies, it is dominated by 46. 3% "trivial" and 11. 3% "impossible" images.
How Well do Feature Visualizations Support Causal Understanding of CNN Activations?
A precise understanding of why units in an artificial network respond to certain stimuli would constitute a big step towards explainable artificial intelligence.
Partial success in closing the gap between human and machine vision
The longstanding distortion robustness gap between humans and CNNs is closing, with the best models now exceeding human feedforward performance on most of the investigated OOD datasets.
Exemplary natural images explain CNN activations better than synthetic feature visualizations
Using a well-controlled psychophysical paradigm, we compare the informativeness of synthetic images \citep{olah2017feature} with a simple baseline visualization, namely exemplary natural images that also strongly activate a specific feature map.
Exemplary Natural Images Explain CNN Activations Better than State-of-the-Art Feature Visualization
Even if only a single reference image is given, synthetic images provide less information than natural images ($65\pm5\%$ vs. $73\pm4\%$).
On the surprising similarities between supervised and self-supervised models
In the light of this recent breakthrough, we here compare self-supervised networks to supervised models and human behaviour.
Natural Images are More Informative for Interpreting CNN Activations than State-of-the-Art Synthetic Feature Visualizations
Using a well-controlled psychophysical paradigm, we compare the informativeness of synthetic images by Olah et al. [45] with a simple baseline visualization, namely natural images that also strongly activate a specific feature map.
Beyond accuracy: quantifying trial-by-trial behaviour of CNNs and humans by measuring error consistency
Here we introduce trial-by-trial error consistency, a quantitative analysis for measuring whether two decision making systems systematically make errors on the same inputs.
Shortcut Learning in Deep Neural Networks
Deep learning has triggered the current rise of artificial intelligence and is the workhorse of today's machine intelligence.
Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Coming
The ability to detect objects regardless of image distortions or weather conditions is crucial for real-world applications of deep learning like autonomous driving.
Ranked #1 on
Robust Object Detection
on MS COCO
Comparison-Based Framework for Psychophysics: Lab versus Crowdsourcing
In this paper we study whether the use of comparison-based (ordinal) data, combined with machine learning algorithms, can boost the reliability of crowdsourcing studies for psychophysics, such that they can achieve performance close to a lab experiment.
ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness
Convolutional Neural Networks (CNNs) are commonly thought to recognise objects by learning increasingly complex representations of object shapes.
Ranked #1 on
Out-of-Distribution Generalization
on ImageNet-W
Generalisation in humans and deep neural networks
We compare the robustness of humans and current convolutional deep neural networks (DNNs) on object recognition under twelve different types of image degradations.
Comparing deep neural networks against humans: object recognition when the signal gets weaker
In addition, we find progressively diverging classification error-patterns between humans and DNNs when the signal gets weaker, indicating that there may still be marked differences in the way humans and current DNNs perform visual object recognition.
