Unifying the Perspectives of NLP and Software Engineering: A Survey on Language Models for Code

1 code implementation • 14 Nov 2023 • Ziyin Zhang, Chaoyu Chen, Bingchang Liu, Cong Liao, Zi Gong, Hang Yu, Jianguo Li, Rui Wang

In this work we systematically review the recent advancements in code processing with language models, covering 50+ models, 30+ evaluation tasks, 170+ datasets, and 800 related works.

MFTCoder: Boosting Code LLMs with Multitask Fine-Tuning

1 code implementation • 4 Nov 2023 • Bingchang Liu, Chaoyu Chen, Cong Liao, Zi Gong, Huan Wang, Zhichao Lei, Ming Liang, Dajun Chen, Min Shen, Hailian Zhou, Hang Yu, Jianguo Li

Code LLMs have emerged as a specialized research field, with remarkable studies dedicated to enhancing model's coding capabilities through fine-tuning on pre-trained models.

Multi-Task Learning

CodeFuse-13B: A Pretrained Multi-lingual Code Large Language Model

no code implementations • 10 Oct 2023 • Peng Di, Jianguo Li, Hang Yu, Wei Jiang, Wenting Cai, Yang Cao, Chaoyu Chen, Dajun Chen, Hongwei Chen, Liang Chen, Gang Fan, Jie Gong, Zi Gong, Wen Hu, Tingting Guo, Zhichao Lei, Ting Li, Zheng Li, Ming Liang, Cong Liao, Bingchang Liu, Jiachen Liu, Zhiwei Liu, Shaojun Lu, Min Shen, Guangpei Wang, Huan Wang, Zhi Wang, Zhaogui Xu, Jiawei Yang, Qing Ye, Gehao Zhang, Yu Zhang, Zelin Zhao, Xunjin Zheng, Hailian Zhou, Lifu Zhu, Xianying Zhu

It is specifically designed for code-related tasks with both English and Chinese prompts and supports over 40 programming languages.

Code Translation Language Modelling +1

MultiCoder: Multi-Programming-Lingual Pre-Training for Low-Resource Code Completion

no code implementations • 19 Dec 2022 • Zi Gong, Yinpeng Guo, Pingyi Zhou, Cuiyun Gao, Yasheng Wang, Zenglin Xu

On the other hand, there are few studies exploring the effects of multi-programming-lingual (MultiPL) pre-training for the code completion, especially the impact on low-resource programming languages.

Code Completion

Source Code Summarization with Structural Relative Position Guided Transformer

1 code implementation • 14 Feb 2022 • Zi Gong, Cuiyun Gao, Yasheng Wang, Wenchao Gu, Yun Peng, Zenglin Xu

We further show that how the proposed SCRIPT captures the structural relative dependencies.

Code Summarization Position +1