Search Results for author:
Found 98 papers, 56 papers with code
A Simple Unsupervised Approach for Coreference Resolution using Rule-based Weak Supervision
Labeled data for the task of Coreference Resolution is a scarce resource, requiring significant human effort.
“Let Your Characters Tell Their Story”: A Dataset for Character-Centric Narrative Understanding
When reading a literary piece, readers often make inferences about various characters’ roles, personalities, relationships, intents, actions, etc.
On the Causal Nature of Sentiment Analysis
Sentiment analysis (SA) aims to identify the sentiment expressed in a text, such as a product review.
Book2Dial: Generating Teacher-Student Interactions from Textbooks for Cost-Effective Development of Educational Chatbots
Educational chatbots are a promising tool for assisting student learning.
Calibrating Large Language Models with Sample Consistency
Accurately gauging the confidence level of Large Language Models' (LLMs) predictions is pivotal for their reliable application.
Competition of Mechanisms: Tracing How Language Models Handle Facts and Counterfactuals
Interpretability research aims to bridge the gap between the empirical success and our scientific understanding of the inner workings of large language models (LLMs).
AFaCTA: Assisting the Annotation of Factual Claim Detection with Reliable LLM Annotators
With the rise of generative AI, automated fact-checking methods to combat misinformation are becoming more and more important.
Scaling the Authoring of AutoTutors with Large Language Models
Large Language Models (LLMs) have found several use cases in education, ranging from automatic question generation to essay evaluation.
Do Language Models Exhibit the Same Cognitive Biases in Problem Solving as Human Learners?
We find evidence that LLMs, with and without instruction-tuning, exhibit human-like biases in both the text-comprehension and the solution-planning steps of the solving process, but not during the final step which relies on the problem's arithmetic expressions (solution execution).
CLadder: Assessing Causal Reasoning in Language Models
Much of the existing work in natural language processing (NLP) focuses on evaluating commonsense causal reasoning in LLMs, thus failing to assess whether a model can perform causal inference in accordance with a set of well-defined formal rules.
RELIC: Investigating Large Language Model Responses using Self-Consistency
To address this challenge, we propose an interactive system that helps users gain insight into the reliability of the generated text.
Navigating the Ocean of Biases: Political Bias Attribution in Language Models via Causal Structures
The rapid advancement of Large Language Models (LLMs) has sparked intense debate regarding their ability to perceive and interpret complex socio-political landscapes.
The ART of LLM Refinement: Ask, Refine, and Trust
In recent years, Large Language Models (LLMs) have demonstrated remarkable generative abilities, but can they judge the quality of their own generations?
Ranked #12 on
Arithmetic Reasoning
on GSM8K
CausalCite: A Causal Formulation of Paper Citations
Evaluating the significance of a paper is pivotal yet challenging for the scientific community.
Towards a Mechanistic Interpretation of Multi-Step Reasoning Capabilities of Language Models
We show that MechanisticProbe is able to detect the information of the reasoning tree from the model's attentions for most examples, suggesting that the LM indeed is going through a process of multi-step reasoning within its architecture in many cases.
A Diachronic Perspective on User Trust in AI under Uncertainty
In a human-AI collaboration, users build a mental model of the AI system based on its reliability and how it presents its decision, e. g. its presentation of system confidence and an explanation of the output.
Let's Synthesize Step by Step: Iterative Dataset Synthesis with Large Language Models by Extrapolating Errors from Small Models
*Data Synthesis* is a promising way to train a small model with very little labeled data.
Agents: An Open-source Framework for Autonomous Language Agents
Recent advances on large language models (LLMs) enable researchers and developers to build autonomous language agents that can automatically solve various tasks and interact with environments, humans, and other agents using natural language interfaces.
A Formal Perspective on Byte-Pair Encoding
Via submodular functions, we prove that the iterative greedy version is a $\frac{1}{{\sigma(\boldsymbol{\mu}^\star)}}(1-e^{-{\sigma(\boldsymbol{\mu}^\star)}})$-approximation of an optimal merge sequence, where ${\sigma(\boldsymbol{\mu}^\star)}$ is the total backward curvature with respect to the optimal merge sequence $\boldsymbol{\mu}^\star$.
Can Large Language Models Infer Causation from Correlation?
In this work, we propose the first benchmark dataset to test the pure causal inference skills of large language models (LLMs).
World Models for Math Story Problems
In this paper, we consolidate previous work on categorizing and representing math story problems and develop MathWorld, which is a graph-based semantic formalism specific for the domain of math story problems.
Infusing Lattice Symmetry Priors in Attention Mechanisms for Sample-Efficient Abstract Geometric Reasoning
As a step towards this goal, we focus on geometry priors and introduce LatFormer, a model that incorporates lattice symmetry priors in attention masks.
Adaptive and Personalized Exercise Generation for Online Language Learning
We train and evaluate our model on real-world learner interaction data from Duolingo and demonstrate that LMs guided by student states can generate superior exercises.
Membership Inference Attacks against Language Models via Neighbourhood Comparison
To investigate whether this fragility provides a layer of safety, we propose and evaluate neighbourhood attacks, which compare model scores for a given sample to scores of synthetically generated neighbour texts and therefore eliminate the need for access to the training data distribution.
A Mechanistic Interpretation of Arithmetic Reasoning in Language Models using Causal Mediation Analysis
Mathematical reasoning in large language models (LMs) has garnered significant attention in recent work, but there is a limited understanding of how these models process and store information related to arithmetic tasks within their architecture.
Linear-Time Modeling of Linguistic Structure: An Order-Theoretic Perspective
We show that these exhaustive comparisons can be avoided, and, moreover, the complexity of such tasks can be reduced to linear by casting the relation between tokens as a partial order over the string.
When Does Aggregating Multiple Skills with Multi-Task Learning Work? A Case Study in Financial NLP
Due to the task difficulty and data scarcity in the Financial NLP domain, we explore when aggregating such diverse skills from multiple datasets with MTL can work.
All Roads Lead to Rome? Exploring the Invariance of Transformers' Representations
Transformer models bring propelling advances in various NLP tasks, thus inducing lots of interpretability research on the learned representations of the models.
MathDial: A Dialogue Tutoring Dataset with Rich Pedagogical Properties Grounded in Math Reasoning Problems
While automatic dialogue tutors hold great potential in making education personalized and more accessible, research on such systems has been hampered by a lack of sufficiently large and high-quality datasets.
RecurrentGPT: Interactive Generation of (Arbitrarily) Long Text
In addition to producing AI-generated content (AIGC), we also demonstrate the possibility of using RecurrentGPT as an interactive fiction that directly interacts with consumers.
Revisiting Automated Topic Model Evaluation with Large Language Models
Topic models are used to make sense of large text collections.
Efficient Prompting via Dynamic In-Context Learning
To achieve this, we train a meta controller that predicts the number of in-context examples suitable for the generalist model to make a good prediction based on the performance-efficiency trade-off for a specific input.
Discourse Centric Evaluation of Machine Translation with a Densely Annotated Parallel Corpus
Several recent papers claim human parity at sentence-level Machine Translation (MT), especially in high-resource languages.
Variational Classification
We present a latent variable model for classification that provides a novel probabilistic interpretation of neural network softmax classifiers.
Beyond Good Intentions: Reporting the Research Landscape of NLP for Social Good
Using state-of-the-art NLP models, we address each of these tasks and use them on the entire ACL Anthology, resulting in a visualization workspace that gives researchers a comprehensive overview of the field of NLP4SG.
Psychologically-Inspired Causal Prompts
In this work, we take sentiment classification as an example and look into the causal relations between the review (X) and sentiment (Y).
Controlled Text Generation with Natural Language Instructions
Large language models generate fluent texts and can follow natural language instructions to solve a wide range of tasks without task-specific training.
Enhancing Textbooks with Visuals from the Web for Improved Learning
We release the dataset of textbooks with an associated image bank to inspire further research in this intersectional area of computer vision and NLP for education.
PWESuite: Phonetic Word Embeddings and Tasks They Facilitate
Mapping words into a fixed-dimensional vector space is the backbone of modern NLP.
Elastic Weight Removal for Faithful and Abstractive Dialogue Generation
We evaluate our method -- using different variants of Flan-T5 as a backbone language model -- on multiple datasets for information-seeking dialogue generation and compare our method with state-of-the-art techniques for faithfulness, such as CTRL, Quark, DExperts, and Noisy Channel reranking.
Strategize Before Teaching: A Conversational Tutoring System with Pedagogy Self-Distillation
Conversational tutoring systems (CTSs) aim to help students master educational material with natural language interaction in the form of a dialog.
Opportunities and Challenges in Neural Dialog Tutoring
Designing dialog tutors has been challenging as it involves modeling the diverse and complex pedagogical strategies employed by human tutors.
Poor Man's Quality Estimation: Predicting Reference-Based MT Metrics Without the Reference
Machine translation quality estimation (QE) predicts human judgements of a translation hypothesis without seeing the reference.
Understanding Stereotypes in Language Models: Towards Robust Measurement and Zero-Shot Debiasing
Generated texts from large pretrained language models have been shown to exhibit a variety of harmful, human-like biases about various demographics.
Distilling Reasoning Capabilities into Smaller Language Models
In this work, we propose an alternative reasoning scheme, Socratic CoT, that learns a decomposition of the original problem into a sequence of subproblems and uses it to guide the intermediate reasoning steps.
Automatic Generation of Socratic Subquestions for Teaching Math Word Problems
On both automatic and human quality evaluations, we find that LMs constrained with desirable question properties generate superior questions and improve the overall performance of a math word problem solver.
Beyond Prompting: Making Pre-trained Language Models Better Zero-shot Learners by Clustering Representations
We further explore the applicability of our clustering approach by evaluating it on 14 datasets with more diverse topics, text lengths, and numbers of classes.
Autoregressive Structured Prediction with Language Models
Recent years have seen a paradigm shift in NLP towards using pretrained language models ({PLM}) for a wide range of tasks.
Ranked #1 on
Relation Extraction
on CoNLL04
(RE+ Micro F1 metric)
A Bilingual Parallel Corpus with Discourse Annotations
The BWB corpus consists of Chinese novels translated by experts into English, and the annotated test set is designed to probe the ability of machine translation systems to model various discourse phenomena.
Investigating the Role of Centering Theory in the Context of Neural Coreference Resolution Systems
Our analysis further shows that contextualized embeddings contain much of the coherence information, which helps explain why CT can only provide little gains to modern neural coreference resolvers which make use of pretrained representations.
Differentially Private Language Models for Secure Data Sharing
To protect the privacy of individuals whose data is being shared, it is of high importance to develop methods allowing researchers and companies to release textual data while providing formal privacy guarantees to its originators.
Adapters for Enhanced Modeling of Multilingual Knowledge and Text
Specifically, we introduce a lightweight adapter set to enhance MLLMs with cross-lingual entity alignment and facts from MLKGs for many languages.
A Causal Framework to Quantify the Robustness of Mathematical Reasoning with Language Models
By grounding the behavioral analysis in a causal graph describing an intuitive reasoning process, we study the behavior of language models in terms of robustness and sensitivity to direct interventions in the input space.
Longtonotes: OntoNotes with Longer Coreference Chains
Ontonotes has served as the most important benchmark for coreference resolution.
When to Make Exceptions: Exploring Language Models as Accounts of Human Moral Judgment
Using a state-of-the-art large language model (LLM) as a basis, we propose a novel moral chain of thought (MORALCOT) prompting strategy that combines the strengths of LLMs with theories of moral reasoning developed in cognitive science to predict human moral judgments.
Learning to Drop Out: An Adversarial Approach to Training Sequence VAEs
In principle, applying variational autoencoders (VAEs) to sequential data offers a method for controlled sequence generation, manipulation, and structured representation learning.
Probing via Prompting
We then examine the usefulness of a specific linguistic property for pre-training by removing the heads that are essential to that property and evaluating the resulting model's performance on language modeling.
A Structured Span Selector
Many natural language processing tasks, e. g., coreference resolution and semantic role labeling, require selecting text spans and making decisions about them.
Original or Translated? A Causal Analysis of the Impact of Translationese on Machine Translation Performance
We show that these two factors have a large causal effect on the MT performance, in addition to the test-model direction mismatch highlighted by existing work on the impact of translationese.
Calibration of Machine Reading Systems at Scale
We show that calibrating such complex systems which contain discrete retrieval and deep reading components is challenging and current calibration techniques fail to scale to these settings.
Slangvolution: A Causal Analysis of Semantic Change and Frequency Dynamics in Slang
We analyze the semantic change and frequency shift of slang words and compare them to those of standard, nonslang words.
Logical Fallacy Detection
In this paper, we propose the task of logical fallacy detection, and provide a new dataset (Logic) of logical fallacies generally found in text, together with an additional challenge set for detecting logical fallacies in climate change claims (LogicClimate).
What Has Been Enhanced in my Knowledge-Enhanced Language Model?
We conduct experiments to verify that our GCS can indeed be used to correctly interpret the KI process, and we use it to analyze two well-known knowledge-enhanced LMs: ERNIE and K-Adapter, and find that only a small amount of factual knowledge is integrated in them.
A Hybrid Neuro-Symbolic approach for Text-Based Games using Inductive Logic Programming
These rules are learned in an online manner and applied with an ASP solver to predict an action for the agent.
Inductive logic programming
Natural Language Understanding
+2
Case-based Reasoning for Better Generalization in Textual Reinforcement Learning
Text-based games (TBG) have emerged as promising environments for driving research in grounded language understanding and studying problems like generalization and sample efficiency.
Out-of-Distribution Generalization
reinforcement-learning
+2
On Learning the Transformer Kernel
In this work we introduce KERNELIZED TRANSFORMER, a generic, scalable, data driven framework for learning the kernel function in Transformers.
Causal Direction of Data Collection Matters: Implications of Causal and Anticausal Learning for NLP
The principle of independent causal mechanisms (ICM) states that generative processes of real world data consist of independent modules which do not influence or inform each other.
"Let Your Characters Tell Their Story": A Dataset for Character-Centric Narrative Understanding
When reading a literary piece, readers often make inferences about various characters' roles, personalities, relationships, intents, actions, etc.
Differentiable Subset Pruning of Transformer Heads
Multi-head attention, a collection of several attention mechanisms that independently attend to different parts of the input, is the key ingredient in the Transformer.
Efficient Text-based Reinforcement Learning by Jointly Leveraging State and Commonsense Graph Representations
Text-based games (TBGs) have emerged as useful benchmarks for evaluating progress at the intersection of grounded language understanding and reinforcement learning (RL).
Self-Supervised Contrastive Learning with Adversarial Perturbations for Defending Word Substitution-based Attacks
In this paper, we present an approach to improve the robustness of BERT language models against word substitution-based adversarial attacks by leveraging adversarial perturbations for self-supervised contrastive learning.
How Good Is NLP? A Sober Look at NLP Tasks through the Lens of Social Impact
We lay the foundations via the moral philosophy definition of social good, propose a framework to evaluate the direct and indirect real-world impact of NLP tasks, and adopt the methodology of global priorities research to identify priority causes for NLP research.
Bird's Eye: Probing for Linguistic Graph Structures with a Simple Information-Theoretic Approach
However, due to the inter-dependence of various phenomena and randomness of training probe models, detecting how these representations encode the rich information in these linguistic graphs remains a challenging problem.
BlonDe: An Automatic Evaluation Metric for Document-level Machine Translation
Standard automatic metrics, e. g. BLEU, are not reliable for document-level MT evaluation.
Deep Clustering of Text Representations for Supervision-free Probing of Syntax
We explore deep clustering of text representations for unsupervised model interpretation and induction of syntax.
Stronger Transformers for Neural Multi-Hop Question Generation
In this work, we introduce a series of strong transformer models for multi-hop question generation, including a graph-augmented transformer that leverages relations between entities in the text.
Text-based RL Agents with Commonsense Knowledge: New Challenges, Environments and Baselines
Text-based games have emerged as an important test-bed for Reinforcement Learning (RL) research, requiring RL agents to combine grounded language understanding with sequential decision making.
Ranked #1 on
Commonsense Reasoning for RL
on commonsense-rl
Text-based RL Agents with Commonsense Knowledge: New Challenges, Environments and Approaches
We introduce a number of RL agents that combine the sequential context with a dynamic graph representation of their beliefs of the world and commonsense knowledge from ConceptNet in different ways.
Knowledge Graph Embedding Compression
Knowledge graph (KG) representation learning techniques that learn continuous embeddings of entities and relations in the KG have become popular in many AI applications.
Enhancing Text-based Reinforcement Learning Agents with Commonsense Knowledge
In this paper, we consider the recent trend of evaluating progress on reinforcement learning technology by using text-based environments and games as evaluation environments.
Discourse in Multimedia: A Case Study in Extracting Geometry Knowledge from Textbooks
At the same time, these help the readers pick up the structure of the discourse and comprehend the conveyed information.
Learning Pipelines with Limited Data and Domain Knowledge: A Study in Parsing Physics Problems
Finally, we also show how Nuts&Bolts can be used to achieve improvements on a relation extraction task and on the end task of answering Newtonian physics problems.
Discourse in Multimedia: A Case Study in Information Extraction
At the same time, these help the readers pick up the structure of the discourse and comprehend the conveyed information.
Contextual Parameter Generation for Universal Neural Machine Translation
We propose a simple modification to existing neural machine translation (NMT) models that enables using a single universal model to translate between multiple languages while allowing for language specific parameterization, and that can also be used for domain adaptation.
Self-Training for Jointly Learning to Ask and Answer Questions
The two tasks of question answering and question generation are usually tackled separately in the NLP literature.
Effective Use of Bidirectional Language Modeling for Transfer Learning in Biomedical Named Entity Recognition
We also show that BiLM weight transfer leads to a faster model training and the pretrained model requires fewer training examples to achieve a particular F1 score.
From Textbooks to Knowledge: A Case Study in Harvesting Axiomatic Knowledge from Textbooks to Solve Geometry Problems
These axioms are then parsed into rules that are used to improve the state-of-the-art in solving geometry problems.
Learning to Solve Geometry Problems from Natural Language Demonstrations in Textbooks
As a case study, we explore the task of learning to solve geometry problems using demonstrative solutions available in textbooks.
Learning Concept Taxonomies from Multi-modal Data
We study the problem of automatically building hypernym taxonomies from textual and visual data.
Science Question Answering using Instructional Materials
We provide a solution for elementary science test using instructional materials.
