Revisiting Out-of-Distribution Detection: A Simple Baseline is Surprisingly Effective

It is an important problem in trustworthy machine learning to recognize out-of-distribution (OOD) inputs which are inputs unrelated to the in-distribution task. Many out-of-distribution detection methods have been suggested in recent years. The goal of this paper is to recognize common objectives as well as to identify the implicit scoring functions of different OOD detection methods. In particular, we show that binary discrimination between in- and (different) out-distributions is equivalent to several different formulations of the OOD detection problem. When trained in a shared fashion with a standard classifier, this binary discriminator reaches an OOD detection performance similar to that of Outlier Exposure. Moreover, we show that the confidence loss which is used by Outlier Exposure has an implicit scoring function which differs in a non-trivial fashion from the theoretically optimal scoring function in the case where training and test out-distribution are the same, but is similar to the one used when training with an extra background class. In practice, when trained in exactly the same way, all these methods perform similarly and reach state-of-the-art OOD detection performance.