Search Results for author:
Found 16 papers, 4 papers with code
Enhancing Trust in LLM-Generated Code Summaries with Calibrated Confidence Scores
There has been a considerable body of research into automated AI-based methods, using Large Language models (LLMs), to generate summaries of code; there also has been quite a bit work on ways to measure the performance of such summarization methods, with special attention paid to how closely these AI-generated summaries resemble a summary a human might have produced.
Studying LLM Performance on Closed- and Open-source Data
We find that performance for C# changes little from OSS --> proprietary code, but does significantly reduce for C++; we find that this difference is attributable to differences in identifiers.
Calibration and Correctness of Language Models for Code
Our contributions will lead to better-calibrated decision-making in the current use of code generated by language models, and offers a framework for future research to further improve calibration methods for generative models in Software Engineering.
Towards Understanding What Code Language Models Learned
To understand the extent to which language models can learn some form of meaning, we investigate their ability to capture semantics of code beyond superficial frequency and co-occurrence.
Better patching using LLM prompting, via Self-Consistency
Large Language models (LLMs) can be induced to solve non-trivial problems with "few-shot" prompts including illustrative problem-solution examples.
Automatic Semantic Augmentation of Language Model Prompts (for Code Summarization)
This approach improves performance in several different settings suggested by prior work, including for two different Large Language Models.
Large Language Models and Simple, Stupid Bugs
We explore the consequences of the Codex generated SStuBs and propose avoidance strategies that suggest the possibility of reducing the production of known, verbatim SStubs, and increase the possibility of producing known, verbatim fixes.
Recommending Root-Cause and Mitigation Steps for Cloud Incidents using Large Language Models
In this work, we do the first large-scale study to evaluate the effectiveness of these models for helping engineers root cause and mitigate production incidents.
Extending Source Code Pre-Trained Language Models to Summarise Decompiled Binaries
While the automated summarisation of decompiled code can help Reverse Engineers understand and analyse binaries, current work mainly focuses on summarising source code, and no suitable dataset exists for this task.
Few-shot training LLMs for project-specific code-summarization
Very large language models (LLMs), such as GPT-3 and Codex have achieved state-of-the-art performance on several natural-language tasks, and show great promise also for code.
NatGen: Generative pre-training by "Naturalizing" source code
Pre-trained Generative Language models (e. g. PLBART, CodeT5, SPT-Code) for source code yielded strong results on several tasks in the past few years, including code generation and translation.
Learning code summarization from a small and local dataset
We compare several models and training approaches, including same-project training, cross-project training, training a model especially designed to be sample efficient (and thus prima facie well-suited for learning in a limited-sample same-project setting) and a maximalist hybrid approach, fine-tuning first on many projects in many languages and then training on the same-project.
Multilingual training for Software Engineering
As a way around such data bottlenecks, we present evidence suggesting that human-written code in different languages (which performs the same function), is rather similar, and particularly preserving of identifier naming patterns; we further present evidence suggesting that identifiers are a very important element of training data for software engineering tasks.
Ranked #5 on
Type prediction
on ManyTypes4TypeScript
Learning Type Annotation: Is Big Data Enough?
ML approaches use different inductive biases, ranging from simple token sequences to complex graphical neural network (GNN) models capturing syntax and semantic relations.
SYNFIX: Automatically Fixing Syntax Errors using Compiler Diagnostics
Beginning programmers struggle with the complex grammar of modern programming languages like Java, and make lot of syntax errors.
Early Prediction for Merged vs Abandoned Code Changes in Modern Code Reviews
In our empirical study on the 146, 612 code changes from the three software projects, we find that (1) The new features like reviewer dimensions that are introduced in PredCR are the most informative.
