no code implementations • 10 May 2024 • Tareen Dawood, Bram Ruijsink, Reza Razavi, Andrew P. King, Esther Puyol-Antón
We evaluate the impact on accuracy and calibration of two types of approach that aim to improve DL classification model calibration: deterministic uncertainty methods (DUM) and uncertainty-aware training.
no code implementations • 29 Aug 2023 • Tareen Dawood, Chen Chen, Baldeep S. Sidhua, Bram Ruijsink, Justin Goulda, Bradley Porter, Mark K. Elliott, Vishal Mehta, Christopher A. Rinaldi, Esther Puyol-Anton, Reza Razavi, Andrew P. King
The best-performing model in terms of both classification accuracy and the most common calibration measure, expected calibration error (ECE) was the Confidence Weight method, a novel approach that weights the loss of samples to explicitly penalise confident incorrect predictions.
no code implementations • 30 Jan 2023 • Tareen Dawood, Emily Chan, Reza Razavi, Andrew P. King, Esther Puyol-Anton
However, in our complex artefact detection task, we saw an improvement in calibration for both a low and higher-capacity model when implementing both the ENN and uncertainty-aware training together, indicating that this approach can offer a promising way to improve calibration in such settings.
no code implementations • 22 Sep 2021 • Tareen Dawood, Chen Chen, Robin Andlauer, Baldeep S. Sidhu, Bram Ruijsink, Justin Gould, Bradley Porter, Mark Elliott, Vishal Mehta, C. Aldo Rinaldi, Esther Puyol-Antón, Reza Razavi, Andrew P. King
Evaluation of predictive deep learning (DL) models beyond conventional performance metrics has become increasingly important for applications in sensitive environments like healthcare.