Vulnerability Detection with Code Language Models: How Far Are We?

1 code implementation • 27 Mar 2024 • Yangruibo Ding, Yanjun Fu, Omniyyah Ibrahim, Chawin Sitawarin, Xinyun Chen, Basel Alomair, David Wagner, Baishakhi Ray, Yizheng Chen

Evaluating code LMs on PrimeVul reveals that existing benchmarks significantly overestimate the performance of these models.

Vulnerability Detection

Jatmo: Prompt Injection Defense by Task-Specific Finetuning

1 code implementation • 29 Dec 2023 • Julien Piet, Maha Alrashed, Chawin Sitawarin, Sizhe Chen, Zeming Wei, Elizabeth Sun, Basel Alomair, David Wagner

Jatmo only needs a task prompt and a dataset of inputs for the task: it uses the teacher model to generate outputs.

Instruction Following

Can LLMs Follow Simple Rules?

1 code implementation • 6 Nov 2023 • Norman Mu, Sarah Chen, Zifan Wang, Sizhe Chen, David Karamardian, Lulwa Aljeraisy, Basel Alomair, Dan Hendrycks, David Wagner

As Large Language Models (LLMs) are deployed with increasing real-world responsibilities, it is important to be able to specify and constrain the behavior of these systems in a reliable manner.