Contrastive Vision-Language Alignment Makes Efficient Instruction Learner

1 code implementation • 29 Nov 2023 • Lizhao Liu, Xinyu Sun, Tianhang Xiang, Zhuangwei Zhuang, Liuren Yin, Mingkui Tan

To address this, existing methods typically train a visual adapter to align the representation between a pre-trained vision transformer (ViT) and the LLM by a generative image captioning loss.

Contrastive Learning Image Captioning +4