Search Results for author:
Found 65 papers, 32 papers with code
NExT: Teaching Large Language Models to Reason about Code Execution
A fundamental skill among human developers is the ability to understand and reason about program execution.
Gemini: A Family of Highly Capable Multimodal Models
This report introduces a new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding.
Ranked #1 on
Multi-task Language Understanding
on MMLU
(using extra training data)
Universal Self-Consistency for Large Language Model Generation
Self-consistency with chain-of-thought prompting (CoT) has demonstrated remarkable performance gains on various challenging tasks, by utilizing multiple reasoning paths sampled from large language models (LLMs).
Training Chain-of-Thought via Latent-Variable Inference
Large language models (LLMs) solve problems more accurately and interpretably when instructed to work out the answer step by step using a ``chain-of-thought'' (CoT) prompt.
ExeDec: Execution Decomposition for Compositional Generalization in Neural Program Synthesis
When writing programs, people have the ability to tackle a new complex task by decomposing it into smaller and more familiar subtasks.
A Probabilistic Framework for Modular Continual Learning
To address this challenge, we develop a modular CL framework, called PICLE, that accelerates search by using a probabilistic model to cheaply compute the fitness of each composition.
Natural Language to Code Generation in Interactive Data Science Notebooks
To measure the performance of AI pair programmers that automatically synthesize programs for those tasks given natural language (NL) intents from users, we build ARCADE, a benchmark of 1082 code generation problems using the pandas data analysis framework in data science notebooks.
A Library for Representing Python Programs as Graphs for Machine Learning
Graph representations of programs are commonly a central element of machine learning for code research.
Repairing Systematic Outliers by Learning Clean Subspaces in VAEs
Seeing as a systematic outlier is a combination of patterns of a clean instance and systematic error patterns, our main insight is that inliers can be modelled by a smaller representation (subspace) in a model than outliers.
Compositional Generalization and Decomposition in Neural Program Synthesis
We first characterize several different axes along which program synthesis methods would be desired to generalize, e. g., length generalization, or the ability to combine known subroutines in new ways that do not occur in the training data.
PaLM: Scaling Language Modeling with Pathways
To further our understanding of the impact of scale on few-shot learning, we trained a 540-billion parameter, densely activated, Transformer language model, which we call Pathways Language Model PaLM.
Ranked #1 on
Coreference Resolution
on Winograd Schema Challenge
CrossBeam: Learning to Search in Bottom-Up Program Synthesis
Many approaches to program synthesis perform a search within an enormous space of programs to find one that satisfies a given specification.
A Bayesian-Symbolic Approach to Reasoning and Learning in Intuitive Physics
In this paper, we propose a Bayesian-symbolic framework (BSP) for physical reasoning and learning that is close to human-level sample-efficiency and accuracy.
Learning Semantic Representations to Verify Hardware Designs
We evaluate Design2Vec on three real-world hardware designs, including an industrial chip used in commercial data centers.
Show Your Work: Scratchpads for Intermediate Computation with Language Models
Large pre-trained language models perform remarkably well on tasks that can be done "in one pass", such as generating realistic text or synthesizing computer programs.
Type Inference as Optimization
We are the first to present a detailed algorithm for NTI that is validated with theorems and proofs.
Program Synthesis with Large Language Models
Our largest models, even without finetuning on a code dataset, can synthesize solutions to 59. 6 percent of the problems from MBPP using few-shot learning with a well-designed prompt.
SpreadsheetCoder: Formula Prediction from Semi-structured Context
In this work, we present the first approach for synthesizing spreadsheet formulas from tabular context, which includes both headers and semi-structured tabular data.
A Bayesian-Symbolic Approach to Learning and Reasoning for Intuitive Physics
As such, learning the laws is then reduced to symbolic regression and Bayesian inference methods are used to obtain the distribution of unobserved properties.
Latent Programmer: Discrete Latent Codes for Program Synthesis
The latent codes are learned using a self-supervised learning principle, in which first a discrete autoencoder is trained on the output sequences, and then the resulting latent codes are used as intermediate targets for the end-to-end sequence prediction task.
Learning Discrete Energy-based Models via Auxiliary-variable Local Exploration
In this paper we propose ALOE, a new algorithm for learning conditional and unconditional EBMs for discrete structured data, where parameter gradients are estimated using a learned sampler that mimics local search.
Learning to Execute Programs with Instruction Pointer Attention Graph Neural Networks
More practically, we evaluate these models on the task of learning to execute partial programs, as might arise if using the model as a heuristic function in program synthesis.
Conditional independence by typing
The resulting program can be seen as a hybrid inference algorithm on the original program, where continuous parameters can be drawn using efficient gradient-based inference methods, while the discrete parameters are inferred using variable elimination.
BUSTLE: Bottom-Up Program Synthesis Through Learning-Guided Exploration
Program synthesis is challenging largely because of the difficulty of search in a large space of programs.
Neural Program Synthesis with a Differentiable Fixer
We present a new program synthesis approach that combines an encoder-decoder based synthesis architecture with a differentiable program fixer.
Global Relational Models of Source Code
By studying a popular, non-trivial program repair task, variable-misuse identification, we explore the relative merits of traditional and hybrid model families for code representation.
SCELMo: Source Code Embeddings from Language Models
Continuous embeddings of tokens in computer programs have been used to support a variety of software development tools, including readability, code search, and program repair.
OptTyper: Probabilistic Type Inference by Optimising Logical and Natural Constraints
OptTyper combines a continuous interpretation of logical constraints derived by classical static analysis of TypeScript code, with natural constraints obtained from a deep learning model, which learns naming conventions for types from a large codebase.
Big Code != Big Vocabulary: Open-Vocabulary Models for Source Code
Statistical language modeling techniques have successfully been applied to large source code corpora, yielding a variety of new software development tools, such as tools for code suggestion, improving readability, and API migration.
Software Engineering
Towards Modular Algorithm Induction
Main consists of a neural controller that interacts with a variable-length input tape and learns to compose modules together with their corresponding argument choices.
Incremental Sampling Without Replacement for Sequence Models
Sampling is a fundamental technique, and sampling without replacement is often desirable when duplicate samples are not beneficial.
Learning to Represent Programs with Property Signatures
We introduce the notion of property signatures, a representation for programs and program specifications meant for consumption by machine learning algorithms.
Learning to Fix Build Errors with Graph2Diff Neural Networks
A diff specifies how to modify the code's abstract syntax tree, represented in the neural network as a sequence of tokens and of pointers to code locations.
Robust Variational Autoencoders for Outlier Detection and Repair of Mixed-Type Data
We show experimentally that not only RVAE performs better than several state-of-the-art methods in cell outlier detection and repair for tabular data, but also that is robust against the initial hyper-parameter selection.
Learning Semantic Annotations for Tabular Data
The usefulness of tabular data such as web tables critically depends on understanding their semantics.
Ranked #1 on
Column Type Annotation
on T2Dv2
How Often Do Single-Statement Bugs Occur? The ManySStuBs4J Dataset
One way to achieve acceptable performance is to focus on classes of simple bugs, such as bugs with single statement fixes, or that match a small set of bug templates.
Software Engineering Programming Languages
Maybe Deep Neural Networks are the Best Choice for Modeling Source Code
We present a new open-vocabulary neural language model for code that is not limited to a fixed vocabulary of identifier names.
Wrangling Messy CSV Files by Detecting Row and Type Patterns
Existing dialect detection approaches are few and non-robust.
Databases E.5; H.2.8
ColNet: Embedding the Semantics of Web Tables for Column Type Prediction
Automatically annotating column types with knowledge base (KB) concepts is a critical task to gain a basic understanding of web tables.
Ranked #1 on
Column Type Annotation
on T2Dv2
(F1 (%) metric)
Probabilistic Programming with Densities in SlicStan: Efficient, Flexible and Deterministic
Stan is a probabilistic programming language that has been increasingly used for real-world scalable projects.
Deep Learning to Detect Redundant Method Comments
To address this problem, we introduce the notion of comment entailment from code, high entailment indicating that a comment's natural language semantics can be inferred directly from the code.
Deep Dungeons and Dragons: Learning Character-Action Interactions from Role-Playing Game Transcripts
An essential aspect to understanding narratives is to grasp the interaction between characters in a story and the actions they take.
Generative Ratio Matching Networks
In this work, we take their insight of using kernels as fixed adversaries further and present a novel method for training deep generative models that does not involve saddlepoint optimization.
Variational Inference In Pachinko Allocation Machines
The Pachinko Allocation Machine (PAM) is a deep topic model that allows representing rich correlation structures among topics by a directed acyclic graph over topics.
HOUDINI: Lifelong Learning as Program Synthesis
We present a neurosymbolic framework for the lifelong learning of algorithmic tasks that mix perception and procedural reasoning.
Interpreting Deep Classifier by Visual Distillation of Dark Knowledge
Interpreting black box classifiers, such as deep networks, allows an analyst to validate a classifier before it is deployed in a high-stakes setting.
GEMSEC: Graph Embedding with Self Clustering
In this paper we propose GEMSEC a graph embedding algorithm which learns a clustering of the nodes simultaneously with computing their features.
Ranked #1 on
Community Detection
on Facebook Athletes
Social and Information Networks
A Survey of Machine Learning for Big Code and Naturalness
We contrast programming languages against natural languages and discuss how these similarities and differences drive the design of probabilistic models.
VEEGAN: Reducing Mode Collapse in GANs using Implicit Variational Learning
Deep generative models provide powerful tools for distributions over complicated manifolds, such as those of natural images.
Autoencoding Variational Inference For Topic Models
A promising approach to address this problem is autoencoding variational Bayes (AEVB), but it has proven diffi- cult to apply to topic models in practice.
Ranked #6 on
Topic Models
on 20NewsGroups
Sequence-to-point learning with neural networks for nonintrusive load monitoring
Interestingly, we systematically show that the convolutional neural networks can inherently learn the signatures of the target appliances, which are automatically added into the model to reduce the identifiability problem.
Tailored Mutants Fit Bugs Better
The results demonstrate that the location selection heuristics produce mutants more closely coupled to real faults for a given budget of mutation operator applications.
Software Engineering
Learning Continuous Semantic Representations of Symbolic Expressions
Combining abstract, symbolic reasoning with continuous neural reasoning is a grand challenge of representation learning.
Clustering with a Reject Option: Interactive Clustering as Bayesian Prior Elicitation
A good clustering can help a data analyst to explore and understand a data set, but what constitutes a good clustering may depend on domain-specific and application-specific criteria.
Clustering with a Reject Option: Interactive Clustering as Bayesian Prior Elicitation
A good clustering can help a data analyst to explore and understand a data set, but what constitutes a good clustering may depend on domain-specific and application-specific criteria.
A Subsequence Interleaving Model for Sequential Pattern Mining
Recent sequential pattern mining methods have used the minimum description length (MDL) principle to define an encoding scheme which describes an algorithm for mining the most compressing patterns in a database.
A Convolutional Attention Network for Extreme Summarization of Source Code
Attention mechanisms in neural networks have proved useful for problems in which the input and output do not have fixed dimension.
Blending LSTMs into CNNs
We consider whether deep convolutional networks (CNNs) can represent decision functions with similar accuracy as recurrent networks such as LSTMs.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Latent Bayesian melding for integrating individual and population models
In many statistical problems, a more coarse-grained model may be suitable for population-level behaviour, whereas a more detailed model is appropriate for accurate modelling of individual behaviour.
A Bayesian Network Model for Interesting Itemsets
Mining itemsets that are the most interesting under a statistical model of the underlying data is a commonly used and well-studied technique for exploratory data analysis, with the most recent interestingness models exhibiting state of the art performance.
Signal Aggregate Constraints in Additive Factorial HMMs, with Application to Energy Disaggregation
Blind source separation problems are difficult because they are inherently unidentifiable, yet the entire goal is to identify meaningful sources.
Semi-Separable Hamiltonian Monte Carlo for Inference in Bayesian Hierarchical Models
Sampling from hierarchical Bayesian models is often difficult for MCMC methods, because of the strong correlations between the model parameters and the hyperparameters.
Scheduled denoising autoencoders
We present a representation learning method that learns features at multiple different levels of scale.
An Introduction to Conditional Random Fields
This tutorial describes conditional random fields, a popular probabilistic method for structured prediction.
