Search Results for author:
Found 14 papers, 6 papers with code
LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
Large Language Models (LLMs) applied to code-related applications have emerged as a prominent field, attracting significant interest from both academia and industry.
The Counterfeit Conundrum: Can Code Language Models Grasp the Nuances of Their Incorrect Generations?
In this work, we focus on these counterfeit samples: programs sampled from a language model that 1) have a high enough log-probability to be generated at a moderate temperature and 2) pass weak correctness checks.
StarCoder 2 and The Stack v2: The Next Generation
Our large model, StarCoder2- 15B, significantly outperforms other models of comparable size.
Ranked #25 on
Code Generation
on MBPP
CRUXEval: A Benchmark for Code Reasoning, Understanding and Execution
The best setup, GPT-4 with chain of thought (CoT), achieves a pass@1 of 75% and 81% on input and output prediction, respectively.
Language Agnostic Code Embeddings
Recently, code language models have achieved notable advancements in addressing a diverse array of essential code comprehension and generation tasks.
LINC: A Neurosymbolic Approach for Logical Reasoning by Combining Language Models with First-Order Logic Provers
Logical reasoning, i. e., deductively inferring the truth value of a conclusion from a set of premises, is an important task for artificial intelligence with wide potential impacts on science, mathematics, and society.
LeanDojo: Theorem Proving with Retrieval-Augmented Language Models
Using this data, we develop ReProver (Retrieval-Augmented Prover): an LLM-based prover augmented with retrieval for selecting premises from a vast math library.
StarCoder: may the source be with you!
The BigCode community, an open-scientific collaboration working on the responsible development of Large Language Models for Code (Code LLMs), introduces StarCoder and StarCoderBase: 15. 5B parameter models with 8K context length, infilling capabilities and fast large-batch inference enabled by multi-query attention.
Ranked #43 on
Code Generation
on MBPP
Certified Interpretability Robustness for Class Activation Mapping
Interpreting machine learning models is challenging but crucial for ensuring the safety of deep networks in autonomous driving systems.
SantaCoder: don't reach for the stars!
The BigCode project is an open-scientific collaboration working on the responsible development of large language models for code.
ObSynth: An Interactive Synthesis System for Generating Object Models from Natural Language Specifications
We introduce ObSynth, an interactive system leveraging the domain knowledge embedded in large language models (LLMs) to help users design object models from high level natural language prompts.
Min-Max Bilevel Multi-objective Optimization with Applications in Machine Learning
We consider a generic min-max multi-objective bilevel optimization problem with applications in robust machine learning such as representation learning and hyperparameter optimization.
The Disagreement Problem in Explainable Machine Learning: A Practitioner's Perspective
To this end, we first conduct interviews with data scientists to understand what constitutes disagreement between explanations generated by different methods for the same model prediction, and introduce a novel quantitative framework to formalize this understanding.
Reproducibility Report: La-MAML: Look-ahead Meta Learning for Continual Learning
The Continual Learning (CL) problem involves performing well on a sequence of tasks under limited compute.
