NExT: Teaching Large Language Models to Reason about Code Execution

no code implementations • 23 Apr 2024 • Ansong Ni, Miltiadis Allamanis, Arman Cohan, Yinlin Deng, Kensen Shi, Charles Sutton, Pengcheng Yin

A fundamental skill among human developers is the ability to understand and reason about program execution.

Program Repair

Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM

1 code implementation • 28 Mar 2024 • Chunqiu Steven Xia, Yinlin Deng, Lingming Zhang

Such limitations inevitably lead us to inquire: Is the leaderboard performance on existing benchmarks reliable and comprehensive enough to measure the program synthesis ability of LLMs?

Code Generation Instruction Following +1

White-box Compiler Fuzzing Empowered by Large Language Models

1 code implementation • 24 Oct 2023 • Chenyuan Yang, Yinlin Deng, Runyu Lu, Jiayi Yao, Jiawei Liu, Reyhaneh Jabbarvand, Lingming Zhang

Nonetheless, prompting LLMs with compiler source-code information remains a missing piece of research in compiler testing.

Code Generation Compiler Optimization

Coverage-Guided Tensor Compiler Fuzzing with Joint IR-Pass Mutation

1 code implementation • 21 Feb 2022 • Jiawei Liu, Yuxiang Wei, Sen yang, Yinlin Deng, Lingming Zhang

Our results show that Tzer substantially outperforms existing fuzzing techniques on tensor compiler testing, with 75% higher coverage and 50% more valuable tests than the 2nd-best technique.