Search Results for author: Richard Shin

Found 20 papers, 8 papers with code

ToolTalk: Evaluating Tool-Usage in a Conversational Setting

1 code implementation15 Nov 2023 Nicholas Farn, Richard Shin

Large language models (LLMs) have displayed massive improvements in reasoning and decision-making skills and can hold natural conversations with users.

Decision Making

BenchCLAMP: A Benchmark for Evaluating Language Models on Syntactic and Semantic Parsing

1 code implementation NeurIPS 2023 Subhro Roy, Sam Thomson, Tongfei Chen, Richard Shin, Adam Pauls, Jason Eisner, Benjamin Van Durme

We introduce BenchCLAMP, a Benchmark to evaluate Constrained LAnguage Model Parsing, that includes context-free grammars for seven semantic parsing datasets and two syntactic parsing datasets with varied output representations, as well as a constrained decoding interface to generate only valid outputs covered by these grammars.

Language Modelling Semantic Parsing +2

Addressing Resource and Privacy Constraints in Semantic Parsing Through Data Augmentation

no code implementations Findings (ACL) 2022 Kevin Yang, Olivia Deng, Charles Chen, Richard Shin, Subhro Roy, Benjamin Van Durme

We introduce a novel setup for low-resource task-oriented semantic parsing which incorporates several constraints that may arise in real-world scenarios: (1) lack of similar datasets/models from a related domain, (2) inability to sample useful logical forms directly from a grammar, and (3) privacy requirements for unlabeled natural utterances.

Data Augmentation Semantic Parsing

Few-Shot Semantic Parsing with Language Models Trained On Code

no code implementations NAACL 2022 Richard Shin, Benjamin Van Durme

Intuitively, such models can more easily output canonical utterances as they are closer to the natural language used for pre-training.

Semantic Parsing

Pruning Pretrained Encoders with a Multitask Objective

no code implementations10 Dec 2021 Patrick Xia, Richard Shin

The sizes of pretrained language models make them challenging and expensive to use when there are multiple desired downstream tasks.

Synthetic Datasets for Neural Program Synthesis

no code implementations ICLR 2019 Richard Shin, Neel Kant, Kavi Gupta, Christopher Bender, Brandon Trabucco, Rishabh Singh, Dawn Song

The goal of program synthesis is to automatically generate programs in a particular language from corresponding specifications, e. g. input-output behavior.

Program Synthesis

RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers

4 code implementations ACL 2020 Bailin Wang, Richard Shin, Xiaodong Liu, Oleksandr Polozov, Matthew Richardson

The generalization challenge lies in (a) encoding the database relations in an accessible way for the semantic parser, and (b) modeling alignment between database columns and their mentions in a given query.

Relation Semantic Parsing +1

Encoding Database Schemas with Relation-Aware Self-Attention for Text-to-SQL Parsers

1 code implementation27 Jun 2019 Richard Shin

When translating natural language questions into SQL queries to answer questions from a database, we would like our methods to generalize to domains and database schemas outside of the training set.

Relation Text-To-SQL

Program Synthesis and Semantic Parsing with Learned Code Idioms

1 code implementation NeurIPS 2019 Richard Shin, Miltiadis Allamanis, Marc Brockschmidt, Oleksandr Polozov

Program synthesis of general-purpose source code from natural language specifications is challenging due to the need to reason about high-level patterns in the target program and low-level implementation details at the same time.

Code Generation Program Synthesis +1

Improving Neural Program Synthesis with Inferred Execution Traces

no code implementations NeurIPS 2018 Richard Shin, Illia Polosukhin, Dawn Song

The task of program synthesis, or automatically generating programs that are consistent with a provided specification, remains a challenging task in artificial intelligence.

Program Synthesis

Parametrized Hierarchical Procedures for Neural Programming

no code implementations ICLR 2018 Roy Fox, Richard Shin, Sanjay Krishnan, Ken Goldberg, Dawn Song, Ion Stoica

Neural programs are highly accurate and structured policies that perform algorithmic tasks by controlling the behavior of a computation mechanism.

Imitation Learning

Learning what to learn in a neural program

no code implementations ICLR 2018 Richard Shin, Dawn Song

Recent work has shown that it is possible to address these issues by using recursion in the Neural Programmer-Interpreter, but this technique requires a verification set which is difficult to construct without knowledge of the internals of the oracle used to generate training data.

JPEG-resistant Adversarial Images

no code implementations NIPS 2017 Workshop on Machine Learning and Computer Security 2017 Richard Shin, Dawn Song

Several papers have explored the use of JPEG compression as a defense against adversarial images.

Making Neural Programming Architectures Generalize via Recursion

no code implementations21 Apr 2017 Jonathon Cai, Richard Shin, Dawn Song

Empirically, neural networks that attempt to learn programs from data have exhibited poor generalizability.

Cannot find the paper you are looking for? You can Submit a new open access paper.