Increasing Trustworthiness of Deep Neural Networks via Accuracy Monitoring

3 Jul 2020Zhihui ShaoJianyi YangShaolei Ren

Inference accuracy of deep neural networks (DNNs) is a crucial performance metric, but can vary greatly in practice subject to actual test datasets and is typically unknown due to the lack of ground truth labels. This has raised significant concerns with trustworthiness of DNNs, especially in safety-critical applications... (read more)

PDF Abstract

Results from the Paper


TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK RESULT BENCHMARK
Image Classification STL-10 Entropy Percentage correct 71.65 # 41
Image Classification STL-10 Accuracy Monitoring Percentage correct 68.62 # 48
Image Classification STL-10 MP Percentage correct 71.05 # 43
Image Classification STL-10 TS Percentage correct 88.03 # 21
Image Classification STL-10 MP* Percentage correct 93.19 # 14

Methods used in the Paper