Search Results for author:
Found 42 papers, 14 papers with code
Think about it! Improving defeasible reasoning by first modeling the question scenario.
Defeasible reasoning is the mode of reasoning where conclusions can be overturned by taking into account new evidence.
proScript: Partially Ordered Scripts Generation
Scripts – prototypical event sequences describing everyday activities – have been shown to help understand narratives by providing expectations, resolving ambiguity, and filling in unstated information.
WorldValuesBench: A Large-Scale Benchmark Dataset for Multi-Cultural Value Awareness of Language Models
The awareness of multi-cultural human values is critical to the ability of language models (LMs) to generate safe and personalized responses.
PROC2PDDL: Open-Domain Planning Representations from Texts
Planning in a text-based environment continues to be a major challenge for AI systems.
Calibrating Large Language Models with Sample Consistency
Accurately gauging the confidence level of Large Language Models' (LLMs) predictions is pivotal for their reliable application.
In-Context Principle Learning from Mistakes
We evaluate LEAP on a wide range of benchmarks, including multi-hop question answering (Hotpot QA), textual QA (DROP), Big-Bench Hard reasoning, and math problems (GSM8K and MATH); in all these benchmarks, LEAP improves the strongest available LLMs such as GPT-3. 5-turbo, GPT-4, GPT-4 turbo and Claude-2. 1.
One Size Does Not Fit All: Customizing Open-Domain Procedures
Our approach is to test several simple multi-LLM-agent architectures for customization, as well as an end-to-end LLM, using a new evaluation set, called CustomPlans, of over 200 WikiHow procedures each with a customization need.
Well begun is half done: Importance of Starting Right in Multi-Step Math Reasoning
We propose two ways in which a smaller model can benefit from initial guidance: 1) asking an LLM for initial guidance, and 2) self-questioning guidance, where the student model can first initiate a question regarding how to start and then continue that chain.
What Makes it Ok to Set a Fire? Iterative Self-distillation of Contexts and Rationales for Disambiguating Defeasible Social and Moral Situations
From this model we distill a high-quality dataset, \delta-Rules-of-Thumb, of 1. 2M entries of contextualizations and rationales for 115K defeasible moral actions rated highly by human annotators 85. 9% to 99. 8% of the time.
CLIN: A Continually Learning Language Agent for Rapid Task Adaptation and Generalization
Language agents have shown some ability to interact with an external environment, e. g., a virtual world such as ScienceWorld, to perform complex tasks, e. g., growing a plant, without the startup costs of reinforcement learning.
Let Me Teach You: Pedagogical Foundations of Feedback for Language Models
In a different world, research in pedagogy has long established several effective feedback models.
Editing Common Sense in Transformers
However, these editing methods have only been evaluated on statements about encyclopedic knowledge with a single correct answer.
Aligning Language Models to User Opinions
An important aspect of developing LLMs that interact with humans is to align models' behavior to their users.
OpenPI2.0: An Improved Dataset for Entity Tracking in Texts
An earlier dataset, OpenPI, provided crowdsourced annotations of entity state changes in text.
RL4F: Generating Natural Language Feedback with Reinforcement Learning for Repairing Model Outputs
Despite their unprecedented success, even the largest language models make mistakes.
Self-Refine: Iterative Refinement with Self-Feedback
Motivated by how humans refine their written text, we introduce Self-Refine, an approach for improving initial outputs from LLMs through iterative feedback and refinement.
Conditional set generation using Seq2seq models
Conditional set generation learns a mapping from an input sequence of tokens to a set.
Memory-assisted prompt editing to improve GPT-3 after deployment
Large LMs such as GPT-3 are powerful, but can commit mistakes that are obvious to humans.
Learning to Repair: Repairing model output errors after deployment using a dynamic memory of feedback
Our goal is for an LM to continue to improve after deployment, without retraining, using feedback from the user.
Interscript: A dataset for interactive learning of scripts through error feedback
We present a new dataset, Interscript, containing user feedback on a deployed model that generates complex everyday tasks.
Think about it! Improving defeasible reasoning by first modeling the question scenario
Defeasible reasoning is the mode of reasoning where conclusions can be overturned by taking into account new evidence.
Could you give me a hint? Generating inference graphs for defeasible reasoning
Defeasible reasoning is the mode of reasoning where conclusions can be overturned by taking into account new evidence.
Improving Neural Model Performance through Natural Language Feedback on Their Explanations
A class of explainable NLP models for reasoning tasks support their decisions by generating free-form or structured explanations, but what happens when these supporting structures contain errors?
proScript: Partially Ordered Scripts Generation via Pre-trained Language Models
Scripts - standardized event sequences describing typical everyday activities - have been shown to help understand narratives by providing expectations, resolving ambiguity, and filling in unstated information.
CURIE: An Iterative Querying Approach for Reasoning About Situations
Recently, models have been shown to predict the effects of unexpected situations, e. g., would cloudy skies help or hinder plant growth?
Information to Wisdom: Commonsense Knowledge Extraction and Compilation
Commonsense knowledge is a foundational cornerstone of artificial intelligence applications.
A Dataset for Tracking Entities in Open Domain Procedural Text
Our solution is a new task formulation where given just a procedural text as input, the task is to generate a set of state change tuples(entity, at-tribute, before-state, after-state)for each step, where the entity, attribute, and state values must be predicted from an open vocabulary.
Do Dogs have Whiskers? A New Knowledge Base of hasPart Relations
We present a new knowledge-base of hasPart relationships, extracted from a large corpus of generic statements.
What-if I ask you to explain: Explaining the effects of perturbations in procedural text
We address the task of explaining the effects of perturbations in procedural text, an important test of process comprehension.
WIQA: A dataset for "What if..." reasoning over procedural text
We introduce WIQA, the first large-scale dataset of "What if..." questions over procedural text.
Everything Happens for a Reason: Discovering the Purpose of Actions in Procedural Text
Our goal is to better comprehend procedural text, e. g., a paragraph about photosynthesis, by not only predicting what happens, but why some actions need to happen before others.
From 'F' to 'A' on the N.Y. Regents Science Exams: An Overview of the Aristo Project
This paper reports unprecedented success on the Grade 8 New York Regents Science Exam, where for the first time a system scores more than 90% on the exam's non-diagram, multiple choice (NDMC) questions.
VISIR: Visual and Semantic Image Label Refinement
CBIR now gains semantic expressiveness by advances in deep-learning-based detection of visual labels.
Know2Look: Commonsense Knowledge for Visual Search
With the rise in popularity of social media, images accompanied by contextual text form a huge section of the web.
Be Consistent! Improving Procedural Text Comprehension using Label Consistency
Our goal is procedural text comprehension, namely tracking how the properties of entities (e. g., their location) change with time given a procedural text (e. g., a paragraph about photosynthesis, a recipe).
Reasoning about Actions and State Changes by Injecting Commonsense Knowledge
Comprehending procedural text, e. g., a paragraph describing photosynthesis, requires modeling actions and the state changes they produce, so that questions about entities at different timepoints can be answered.
Tracking State Changes in Procedural Text: A Challenge Dataset and Models for Process Paragraph Comprehension
The new dataset, ProPara, is the first to contain natural (rather than machine-generated) text about a changing world along with a full annotation of entity states (location and existence) during those changes (81k datapoints).
Ranked #4 on
Procedural Text Understanding
on ProPara
What Happened? Leveraging VerbNet to Predict the Effects of Actions in Procedural Text
To supply this knowledge, we leverage VerbNet to build a rulebase (called the Semantic Lexicon) of the preconditions and effects of actions, and use it along with commonsense knowledge of persistence to answer questions about change.
Learning Language-Visual Embedding for Movie Understanding with Natural-Language
We evaluate our models on large scale LSMDC16 movie dataset for two tasks: 1) Standard Ranking for video annotation and retrieval 2) Our proposed movie multiple-choice test.
Ranked #39 on
Video Retrieval
on MSR-VTT
Movie Description
In addition we also collected and aligned movie scripts used in prior work and compare the two sources of descriptions.
A Dataset for Movie Description
In this work we propose a novel dataset which contains transcribed DVS, which is temporally aligned to full length HD movies.
