Search Results for author:
Found 7 papers, 6 papers with code
EVA-CLIP-18B: Scaling CLIP to 18 Billion Parameters
Scaling up contrastive language-image pretraining (CLIP) is critical for empowering both vision and multimodal models.
Ranked #1 on
Zero-Shot Transfer Image Classification
on SUN
Image Classification
Zero-Shot Transfer Image Classification
Generative Multimodal Models are In-Context Learners
The human ability to easily solve multimodal tasks in context (i. e., with only a few demonstrations or simple instructions), is what current multimodal systems have largely struggled to imitate.
Ranked #21 on
Visual Question Answering
on MM-Vet
CapsFusion: Rethinking Image-Text Data at Scale
To provide higher-quality and more scalable multimodal pretraining data, we propose CapsFusion, an advanced framework that leverages large language models to consolidate and refine information from both web-based image-text pairs and synthetic captions.
Multimodal Molecular Pretraining via Modality Blending
Self-supervised learning has recently gained growing interest in molecular modeling for scientific tasks such as AI-assisted drug discovery.
Generative Pretraining in Multimodality
We present Emu, a Transformer-based multimodal foundation model, which can seamlessly generate images and texts in multimodal context.
Ranked #1 on
Visual Question Answering
on VQA v2
Multimodal Federated Learning via Contrastive Representation Ensemble
In this work, we propose Contrastive Representation Ensemble and Aggregation for Multimodal FL (CreamFL), a multimodal federated learning framework that enables training larger server models from clients with heterogeneous model architectures and data modalities, while only communicating knowledge on public dataset.
Adversarial Contrastive Learning via Asymmetric InfoNCE
Contrastive learning (CL) has recently been applied to adversarial learning tasks.
