Code Generation
337 papers with code • 17 benchmarks • 43 datasets
Code Generation is an important field to predict explicit code or program structure from multimodal data sources such as incomplete code, programs in another programming language, natural language descriptions or execution examples. Code Generation tools can assist the development of automatic programming tools to improve programming productivity.
Source: Deep Learning for Source Code Modeling and Generation
Image source: Measuring Coding Challenge Competence With APPS
Benchmarks
These leaderboards are used to track progress in Code Generation
| Trend | Dataset | Best Model | Paper | Code | Compare |
|---|---|---|---|---|---|
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
Libraries
Use these libraries to find Code Generation models and implementations
HumanEval
MBPP
WikiSQL
CodeXGLUE
APPS
DS-1000
Django
Subtasks
Latest papers
Self-Organized Agents: A LLM Multi-Agent Framework toward Ultra Large-Scale Code Generation and Optimization
To tackle this challenge, we propose Self-Organized multi-Agent framework (SoA), a novel multi-agent framework that enables the scalable and efficient generation and optimization of large-scale code.
EvoCodeBench: An Evolving Code Generation Benchmark Aligned with Real-World Code Repositories
Existing benchmarks demonstrate poor alignment with real-world code repositories and are insufficient to evaluate the coding abilities of LLMs.
CodeBenchGen: Creating Scalable Execution-based Code Generation Benchmarks
To demonstrate the complexity and solvability of examples in Exec-CSN, we present a human study demonstrating that 81. 3% of the examples can be solved by humans and 61% are rated as "requires effort to solve".
Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM
Such limitations inevitably lead us to inquire: Is the leaderboard performance on existing benchmarks reliable and comprehensive enough to measure the program synthesis ability of LLMs?
CYCLE: Learning to Self-Refine the Code Generation
Pre-trained code language models have achieved promising performance in code generation and improved the programming efficiency of human developers.
Diffusion-based Aesthetic QR Code Generation via Scanning-Robust Perceptual Guidance
In this paper, we introduce a novel diffusion-model-based aesthetic QR code generation pipeline, utilizing pre-trained ControlNet and guided iterative refinement via a novel classifier guidance (SRG) based on the proposed Scanning-Robust Loss (SRL) tailored with QR code mechanisms, which ensures both aesthetics and scannability.
Exploring Language Model's Code Generation Ability with Auxiliary Functions
However, our analysis also reveals the model's underutilized behavior to call the auxiliary function, suggesting the future direction to enhance their implementation by eliciting the auxiliary function call ability encoded in the models.
DevBench: A Comprehensive Benchmark for Software Development
Recent advancements in large language models (LLMs) have significantly enhanced their coding capabilities.
CleanAgent: Automating Data Standardization with LLM-based Agents
Data standardization is a crucial part in data science life cycle.
Bugs in Large Language Models Generated Code: An Empirical Study
The bug patterns are presented in the form of a taxonomy.
