no code implementations • 12 Nov 2023 • Kristof Meding, Thilo Hagendorff
Our paper intends to serve as a guidance for discussions within the fair ML community to prevent or reduce the misuse of fairness metrics, and thus reduce overall harm from ML applications.
1 code implementation • 12 Oct 2021 • Kristof Meding, Luca M. Schulze Buschoff, Robert Geirhos, Felix A. Wichmann
We find that the ImageNet validation set, among others, suffers from dichotomous data difficulty (DDD): For the range of investigated models and their accuracies, it is dominated by 46. 0% "trivial" and 11. 5% "impossible" images (beyond label errors).
no code implementations • ICLR 2022 • Kristof Meding, Luca M. Schulze Buschoff, Robert Geirhos, Felix A. Wichmann
We find that the ImageNet validation set, among others, suffers from dichotomous data difficulty (DDD): For the range of investigated models and their accuracies, it is dominated by 46. 0% ``trivial'' and 11. 5% ``impossible'' images (beyond label errors).
no code implementations • NeurIPS Workshop ImageNet_PPF 2021 • Kristof Meding, Luca M. Schulze Buschoff, Robert Geirhos, Felix A. Wichmann
We find that the ImageNet validation set suffers from dichotomous data difficulty (DDD): For the range of investigated models and their accuracies, it is dominated by 46. 3% "trivial" and 11. 3% "impossible" images.
1 code implementation • NeurIPS 2020 • Robert Geirhos, Kristof Meding, Felix A. Wichmann
Here we introduce trial-by-trial error consistency, a quantitative analysis for measuring whether two decision making systems systematically make errors on the same inputs.
no code implementations • 8 Jun 2020 • Thilo Hagendorff, Kristof Meding
For this purpose, we have not only carried out an informed ethical analysis of the field, but have inspected all papers of the main ML conferences NeurIPS, CVPR, and ICML of the last 5 years - almost 11, 000 papers in total.
no code implementations • NeurIPS 2019 • Kristof Meding, Dominik Janzing, Bernhard Schölkopf, Felix A. Wichmann
We employ a so-called frozen noise paradigm enabling us to compare human performance with four different algorithms on a trial-by-trial basis: A causal inference algorithm exploiting the dependence structure of additive noise terms, a neurally inspired network, a Bayesian ideal observer model as well as a simple heuristic.