Efficient Out-of-Distribution Detection with Prototypical Semi-Supervised Learning and Foundation Models

This paper describes PAWS-VMK, an improved approach to prototypical semi-supervised learning in the field of computer vision, specifically designed to utilize a frozen foundation model as the neural network backbone. This method outperforms previous results in semi-supervised learning and out-of-distribution (OOD) detection, improving upon the Predicting View-Assignments With Support Samples (PAWS) semi-supervised learning method. We introduce (1) parametric von-Mises Fisher Stochastic Neighbour Embedding (vMF-SNE) to pretrain the projection head using the high-quality embeddings of the foundation model; (2) a MixMatch inspired loss, where predictions across multiple views are averaged to provide a more reliable supervision signal compared to the consistency loss used in PAWS and (3) simple $k$-Means prototype selection (SKMPS), a technique that provides superior performance to other unsupervised label selection approaches in this context. PAWS-VMK sets new benchmarks in semi-supervised learning for CIFAR-10 (99.2%) and CIFAR-100 (89.8%) with four labelled instances per class, and Food-101 (90.1%) with two labelled instances per class. We also observe that PAWS-VMK can efficiently detect OOD samples in a manner that is competitive with specialised methods specifically designed for this purpose, achieving 93.1/98.0 and 95.2/96.3 on the CIFAR-10 and CIFAR-100 OpenOOD benchmarks.