EVA-CLIP-18B: Scaling CLIP to 18 Billion Parameters
Scaling up contrastive language-image pretraining (CLIP) is critical for empowering both vision and multimodal models. We present EVA-CLIP-18B, the largest and most powerful open-source CLIP model to date, with 18-billion parameters. With only 6-billion training samples seen, EVA-CLIP-18B achieves an exceptional 80.7% zero-shot top-1 accuracy averaged across 27 widely recognized image classification benchmarks, outperforming its forerunner EVA-CLIP (5-billion parameters) and other open-source CLIP models by a large margin. Remarkably, we observe a consistent performance improvement with the model size scaling of EVA-CLIP, despite maintaining a constant training dataset of 2-billion image-text pairs from LAION-2B and COYO-700M. This dataset is openly available and much smaller than the in-house datasets (e.g., DFN-5B, WebLI-10B) employed in other state-of-the-art CLIP models. EVA-CLIP-18B demonstrates the potential of EVA-style weak-to-strong visual model scaling. With our model weights made publicly available, we hope to facilitate future research in vision and multimodal foundation models.
PDF AbstractCode
CIFAR-10
ImageNet
MS COCO
CIFAR-100
MNIST
UCF101
ModelNet
Flickr30k
DTD
Food-101
EuroSAT
ImageNet-R
ImageNet-A
ScanObjectNN
ImageNet-Sketch
ObjectNet
Kinetics-600
RESISC45
Kinetics-700
PCam
Birdsnap
Results from the Paper
| Task | Dataset | Model | Metric Name | Metric Value | Global Rank | Uses Extra Training Data |
Benchmark |
|---|---|---|---|---|---|---|---|
| Zero-Shot Transfer Image Classification | Food-101 | EVA-CLIP-18B | Top 1 Accuracy | 95.8 | # 2 | ||
| Zero-Shot Transfer Image Classification | ImageNet | EVA-CLIP-18B | Accuracy (Private) | 83.8 | # 9 | ||
| Zero-Shot Transfer Image Classification | ImageNet-A | EVA-CLIP-18B | Accuracy (Private) | 87.3 | # 4 | ||
| Zero-Shot Transfer Image Classification | ImageNet-R | EVA-CLIP-18B | Accuracy | 95.7 | # 5 | ||
| Zero-Shot Transfer Image Classification | ImageNet-Sketch | EVA-CLIP-18B | Accuracy (Private) | 74.7 | # 4 | ||
| Zero-Shot Transfer Image Classification | ImageNet V2 | EVA-CLIP-18B | Accuracy (Private) | 77.9 | # 7 | ||
| Zero-Shot Transfer Image Classification | ObjectNet | EVA-CLIP-18B | Accuracy (Private) | 82.2 | # 4 | ||
| Zero-Shot Transfer Image Classification | SUN | EVA-CLIP-18B | Accuracy | 77.7 | # 1 |
