Search Results for author: Koushik Sen

Found 17 papers, 10 papers with code

LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code

no code implementations12 Mar 2024 Naman jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, Ion Stoica

Large Language Models (LLMs) applied to code-related applications have emerged as a prominent field, attracting significant interest from both academia and industry.

Code Generation

The Counterfeit Conundrum: Can Code Language Models Grasp the Nuances of Their Incorrect Generations?

no code implementations29 Feb 2024 Alex Gu, Wen-Ding Li, Naman jain, Theo X. Olausson, Celine Lee, Koushik Sen, Armando Solar-Lezama

In this work, we focus on these counterfeit samples: programs sampled from a language model that 1) have a high enough log-probability to be generated at a moderate temperature and 2) pass weak correctness checks.

Code Generation Language Modelling

CodeScholar: Growing Idiomatic Code Examples

1 code implementation23 Dec 2023 Manish Shetty, Koushik Sen, Ion Stoica

A tool that could generate realistic, idiomatic, and contextual usage examples for one or more APIs would be immensely beneficial to developers.

Program Synthesis

DSPy Assertions: Computational Constraints for Self-Refining Language Model Pipelines

1 code implementation20 Dec 2023 Arnav Singhvi, Manish Shetty, Shangyin Tan, Christopher Potts, Koushik Sen, Matei Zaharia, Omar Khattab

We integrate our constructs into the recent DSPy programming model for LMs, and present new strategies that allow DSPy to compile programs with LM Assertions into more reliable and accurate systems.

Language Modelling Prompt Engineering +2

LLM-Assisted Code Cleaning For Training Accurate Code Generators

no code implementations25 Nov 2023 Naman jain, Tianjun Zhang, Wei-Lin Chiang, Joseph E. Gonzalez, Koushik Sen, Ion Stoica

In this work, we investigate data quality for code and find that making the code more structured and readable leads to improved code generation performance of the system.

Code Generation

SlimFit: Memory-Efficient Fine-Tuning of Transformer-based Models Using Training Dynamics

no code implementations29 May 2023 Arash Ardakani, Altan Haan, Shangyin Tan, Doru Thom Popovici, Alvin Cheung, Costin Iancu, Koushik Sen

This allows SlimFit to freeze up to 95% of layers and reduce the overall on-device GPU memory usage of transformer-based models such as ViT and BERT by an average of 2. 2x, across different NLP and CV benchmarks/datasets such as GLUE, SQuAD 2. 0, CIFAR-10, CIFAR-100 and ImageNet with an average degradation of 0. 2% in accuracy.

Quantization Scheduling

Benchmarking Language Models for Code Syntax Understanding

1 code implementation26 Oct 2022 Da Shen, Xinyun Chen, Chenguang Wang, Koushik Sen, Dawn Song

Our key observation is that existing language models pretrained on code still lack the understanding of code syntax.

Benchmarking

LGV: Boosting Adversarial Example Transferability from Large Geometric Vicinity

1 code implementation26 Jul 2022 Martin Gubri, Maxime Cordy, Mike Papadakis, Yves Le Traon, Koushik Sen

We propose transferability from Large Geometric Vicinity (LGV), a new technique to increase the transferability of black-box adversarial attacks.

Adversarial Attack

Efficient and Transferable Adversarial Examples from Bayesian Neural Networks

1 code implementation10 Nov 2020 Martin Gubri, Maxime Cordy, Mike Papadakis, Yves Le Traon, Koushik Sen

An established way to improve the transferability of black-box evasion attacks is to craft the adversarial examples on an ensemble-based surrogate to increase diversity.

Adversarial Attack Bayesian Inference

QFAST: Quantum Synthesis Using a Hierarchical Continuous Circuit Space

3 code implementations9 Mar 2020 Ed Younis, Koushik Sen, Katherine Yelick, Costin Iancu

We present QFAST, a quantum synthesis tool designed to produce short circuits and to scale well in practice.

Quantum Physics

When Deep Learning Met Code Search

2 code implementations9 May 2019 Jose Cambronero, Hongyu Li, Seohyun Kim, Koushik Sen, Satish Chandra

Our evaluation shows that: 1. adding supervision to an existing unsupervised technique can improve performance, though not necessarily by much; 2. simple networks for supervision can be more effective that more sophisticated sequence-based networks for code search; 3. while it is common to use docstrings to carry out supervision, there is a sizeable gap between the effectiveness of docstrings and a more query-appropriate supervision corpus.

Code Search Natural Language Queries

Aroma: Code Recommendation via Structural Code Search

2 code implementations4 Dec 2018 Sifei Luan, Di Yang, Koushik Sen, Satish Chandra

Such a tool could help programmers to extend partially written code snippets to completely implement necessary functionality, help to discover extensions to the partial code which are commonly done by other programmers, help to cross-check against similar code written by other programmers, or help to add extra code which would avoid common mistakes and errors.

Software Engineering

Context2Name: A Deep Learning-Based Approach to Infer Natural Variable Names from Usage Contexts

no code implementations31 Aug 2018 Rohan Bavishi, Michael Pradel, Koushik Sen

Most of the JavaScript code deployed in the wild has been minified, a process in which identifier names are replaced with short, arbitrary and meaningless names.

DeepBugs: A Learning Approach to Name-based Bug Detection

2 code implementations30 Apr 2018 Michael Pradel, Koushik Sen

We formulate bug detection as a binary classification problem and train a classifier that distinguishes correct from incorrect code.

Software Engineering Programming Languages

FairFuzz: Targeting Rare Branches to Rapidly Increase Greybox Fuzz Testing Coverage

no code implementations20 Sep 2017 Caroline Lemieux, Koushik Sen

However, AFL remains limited in the depth of program coverage it achieves, in particular because it does not consider which parts of program inputs should not be mutated in order to maintain deep program coverage.

Software Engineering Cryptography and Security

Cannot find the paper you are looking for? You can Submit a new open access paper.