no code implementations • 26 Apr 2024 • Yuhang Huang, Zihan Wu, Chongyang Gao, Jiawei Peng, Xu Yang
Large Vision-Language Models (LVLMs) are gaining traction for their remarkable ability to process and integrate visual and textual data.
1 code implementation • 13 Feb 2024 • Chongyang Gao, Kezhen Chen, Jinmeng Rao, Baochen Sun, Ruibo Liu, Daiyi Peng, Yawen Zhang, Xiaoyuan Guo, Jie Yang, VS Subrahmanian
In this paper, we introduce a novel parameter-efficient MoE method, \textit{\textbf{M}oE-L\textbf{o}RA with \textbf{L}ayer-wise Expert \textbf{A}llocation (MoLA)} for Transformer-based models, where each model layer has the flexibility to employ a varying number of LoRA experts.
1 code implementation • 4 Dec 2023 • Li Li, Jiawei Peng, Huiyi Chen, Chongyang Gao, Xu Yang
Inspired by the success of Large Language Models in dealing with new tasks via In-Context Learning (ICL) in NLP, researchers have also developed Large Vision-Language Models (LVLMs) with ICL capabilities.
1 code implementation • ICCV 2023 • Weiyi Wu, Chongyang Gao, Joseph DiPalma, Soroush Vosoughi, Saeed Hassanpour
This framework aims for transferable representation learning and semantically meaningful clustering by synergizing invariance loss and clustering loss in WSI analysis.
1 code implementation • NeurIPS 2023 • Yiren Jian, Chongyang Gao, Soroush Vosoughi
We present a novel methodology aimed at optimizing the application of frozen large language models (LLMs) for resource-intensive vision-language (VL) pre-training.
no code implementations • 1 Jun 2023 • Lili Wang, Chenghan Huang, Chongyang Gao, Weicheng Ma, Soroush Vosoughi
In the pursuit of accurate and scalable quantitative methods for financial market analysis, the focus has shifted from individual stock models to those capturing interrelations between companies and their stocks.
1 code implementation • 13 Feb 2023 • Yiren Jian, Chongyang Gao, Chen Zeng, Yunjie Zhao, Soroush Vosoughi
Our findings indicate that the learned structural patterns of proteins can be transferred to RNAs, opening up potential new avenues for research.
1 code implementation • 4 Oct 2022 • Xu Yang, Hanwang Zhang, Chongyang Gao, Jianfei Cai
This is because the language is only partially observable, for which we need to dynamically collocate the modules during the process of image captioning.
1 code implementation • 20 Sep 2022 • Yiren Jian, Chongyang Gao, Soroush Vosoughi
This indicates that Transformer models are able to generalize better by doing a similar task (i. e., clustering) with unpaired examples from different modalities in a multi-task fashion.
1 code implementation • NAACL 2022 • Yiren Jian, Chongyang Gao, Soroush Vosoughi
Few-shot language learners adapt knowledge from a pre-trained model to recognize novel classes from a few-labeled sentences.
1 code implementation • NAACL 2022 • Yiren Jian, Chongyang Gao, Soroush Vosoughi
Following this line of work, we present a contrastive learning framework that clusters inputs from the same class for better generality of models trained with only limited examples.
1 code implementation • ICLR 2022 • Ruibo Liu, Chongyang Gao, Chenyan Jia, Guangxuan Xu, Soroush Vosoughi
The performance of existing text style transfer models is severely limited by the non-parallel datasets on which the models are trained.
1 code implementation • ICLR 2022 • Ruibo Liu, Guoqing Zheng, Shashank Gupta, Radhika Gaonkar, Chongyang Gao, Soroush Vosoughi, Milad Shokouhi, Ahmed Hassan Awadallah
Hence, they tend to suffer from counterfactual or hallucinatory generation when used in knowledge-intensive natural language generation (NLG) tasks.
Ranked #2 on Question Answering on KILT: ELI5
1 code implementation • 5 Oct 2021 • Yiren Jian, Chongyang Gao
Previous work has shown that the performance of a semantic segmentation model can be improved by training jointly with real and synthetic examples with a proper weighting on the synthetic data.
no code implementations • ICCV 2021 • Xu Yang, Chongyang Gao, Hanwang Zhang, Jianfei Cai
We propose an Auto-Parsing Network (APN) to discover and exploit the input data's hidden tree structures for improving the effectiveness of the Transformer-based vision-language systems.
no code implementations • 18 Jun 2021 • Lili Wang, Chongyang Gao, Chenghan Huang, Ruibo Liu, Weicheng Ma, Soroush Vosoughi
A common type of network is the heterogeneous network, where the nodes (and edges) can be of different types.
no code implementations • EMNLP (WNUT) 2020 • Lili Wang, Chongyang Gao, Jason Wei, Weicheng Ma, Ruibo Liu, Soroush Vosoughi
The field of NLP has seen unprecedented achievements in recent years.
no code implementations • ACM International Conference on Multimedia 2020 • Yang, Xu, Chongyang Gao, Hanwang Zhang, and Jianfei Cai
We propose irredundant attention in SSG-RNN to improve the possibility of abstracting topics from rarely described sub-graphs and inheriting attention in WSG-RNN to generate more grounded sentences with the abstracted topics, both of which give rise to more distinctive paragraphs.