Search Results for author:
Found 60 papers, 25 papers with code
Explaining Why: How Instructions and User Interfaces Impact Annotator Rationales When Labeling Text Data
In the context of data labeling, NLP researchers are increasingly interested in having humans select rationales, a subset of input tokens relevant to the chosen label.
Human-Centered Evaluation of Explanations
The NLP community are increasingly interested in providing explanations for NLP models to help people make sense of model behavior and potentially improve human interaction with models.
Characterizing Multimodal Long-form Summarization: A Case Study on Financial Reports
We propose a computational framework for characterizing multimodal long-form summarization and investigate the behavior of Claude 2. 0/2. 1, GPT-4/3. 5, and Command.
Hypothesis Generation with Large Language Models
We focus on hypothesis generation based on data (i. e., labeled examples).
OpenHEXAI: An Open-Source Framework for Human-Centered Evaluation of Explainable Machine Learning
However, properly evaluating the effectiveness of the XAI methods inevitably requires the involvement of human subjects, and conducting human-centered benchmarks is challenging in a number of ways: designing and implementing user studies is complex; numerous design choices in the design space of user study lead to problems of reproducibility; and running user studies can be challenging and even daunting for machine learning researchers.
Quantifying the Uniqueness of Donald Trump in Presidential Discourse
Does Donald Trump speak differently from other presidents?
Clinical Notes Reveal Physician Fatigue
Finally, the model indicates that notes written about Black and Hispanic patients have 12% and 21% higher predicted fatigue than Whites -- larger than overnight vs. daytime differences.
Pragmatic Radiology Report Generation
When pneumonia is not found on a chest X-ray, should the report describe this negative observation or omit it?
Ecologically Valid Explanations for Label Variation in NLI
Human label variation, or annotation disagreement, exists in many natural language processing (NLP) tasks, including natural language inference (NLI).
FLamE: Few-shot Learning from Natural Language Explanations
Natural language explanations have the potential to provide rich information that in principle guides model reasoning.
Language of Bargaining
Leveraging an established exercise in negotiation education, we build a novel dataset for studying how the use of language shapes bilateral bargaining.
Language Models Can Improve Event Prediction by Few-Shot Abductive Reasoning
Large language models have shown astonishing performance on a wide range of reasoning tasks.
Entity-Based Evaluation of Political Bias in Automatic Summarization
Growing literature has shown that NLP systems may encode social biases; however, the political bias of summarization models remains relatively unknown.
Causal Reasoning and Large Language Models: Opening a New Frontier for Causality
The causal capabilities of large language models (LLMs) is a matter of significant debate, with critical implications for the use of LLMs in societally impactful domains such as medicine, science, law, and policy.
Understanding and Predicting Human Label Variation in Natural Language Inference through Explanation
Human label variation (Plank 2022), or annotation disagreement, exists in many natural language processing (NLP) tasks.
Learning Human-Compatible Representations for Case-Based Decision Support
Despite the promising performance of supervised learning, representations learned by supervised models may not align well with human intuitions: what models consider as similar examples can be perceived as distinct by humans.
Selective Explanations: Leveraging Human Input to Align Explainable AI
As a result, current XAI techniques are often found to be hard to use and lack effectiveness.
1st Workshop on Maritime Computer Vision (MaCVi) 2023: Challenge Results
The 1$^{\text{st}}$ Workshop on Maritime Computer Vision (MaCVi) 2023 focused on maritime computer vision for Unmanned Aerial Vehicles (UAV) and Unmanned Surface Vehicle (USV), and organized several subchallenges in this domain: (i) UAV-based Maritime Object Detection, (ii) UAV-based Maritime Object Tracking, (iii) USV-based Maritime Obstacle Segmentation and (iv) USV-based Maritime Obstacle Detection.
Active Example Selection for In-Context Learning
For GPT-2, our learned policies demonstrate strong abilities of generalizing to unseen tasks in training, with a $5. 8\%$ improvement on average.
Probing Classifiers are Unreliable for Concept Removal and Detection
Even under the most favorable conditions for learning a probing classifier when a concept's relevant features in representation space alone can provide 100% accuracy, we prove that a probing classifier is likely to use non-concept features and thus post-hoc or adversarial methods will fail to remove the concept correctly.
Learning to Ignore Adversarial Attacks
Despite the strong performance of current NLP models, they can be brittle against adversarial attacks.
Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation
Despite impressive performance in many benchmark datasets, AI models can still make mistakes, especially among out-of-distribution examples.
Machine Explanations and Human Understanding
Our key result is that without assumptions about task-specific intuitions, explanations may potentially improve human understanding of model decision boundary, but they cannot improve human understanding of task decision boundary or model error.
Rethinking Explainability as a Dialogue: A Practitioner's Perspective
Overall, we hope our work serves as a starting place for researchers and engineers to design interactive explainability systems.
Towards a Science of Human-AI Decision Making: A Survey of Empirical Studies
Besides developing AI technologies for this purpose, the emerging field of human-AI decision making must embrace empirical approaches to form a foundational understanding of how humans interact and work with AI to make decisions.
What to Learn, and How: Toward Effective Learning from Rationales
Learning from rationales seeks to augment model prediction accuracy using human-annotated rationales (i. e. subsets of input tokens) that justify their chosen labels, often in the form of intermediate or multitask supervision.
Investigating the Effect of Natural Language Explanations on Out-of-Distribution Generalization in Few-shot NLI
Although neural models have shown strong performance in datasets such as SNLI, they lack the ability to generalize out-of-distribution (OOD).
Decision-Focused Summarization
To evaluate our method (DecSum), we build a testbed where the task is to summarize the first ten reviews of a restaurant in support of predicting its future rating on Yelp.
On the Diversity and Limits of Human Explanations
A growing effort in NLP aims to build datasets of human explanations.
On Positivity Bias in Negative Reviews
Prior work has revealed that positive words occur more frequently than negative words in human expressions, which is typically attributed to positivity bias, a tendency for people to report positive views of reality.
Using Transformers to Provide Teachers with Personalized Feedback on their Classroom Discourse: The TalkMoves Application
TalkMoves is an innovative application designed to support K-12 mathematics teachers to reflect on, and continuously improve their instructional practices.
Understanding the Effect of Out-of-distribution Examples and Interactive Explanations on Human-AI Decision Making
Although AI holds promise for improving human decision making in societally critical domains, it remains an open question how human-AI teams can reliably outperform AI alone and human alone in challenging prediction tasks (also known as complementary performance).
Towards Unifying Feature Attribution and Counterfactual Explanations: Different Means to the Same End
In addition, by restricting the features that can be modified for generating counterfactual examples, we find that the top-k features from LIME or SHAP are often neither necessary nor sufficient explanations of a model's prediction.
Evaluating and Characterizing Human Rationales
Two main approaches for evaluating the quality of machine-generated rationales are: 1) using human rationales as a gold standard; and 2) automated metrics based on how rationales affect model behavior.
Characterizing the Value of Information in Medical Notes
Machine learning models depend on the quality of input data.
Harnessing Explanations to Bridge AI and Humans
Machine learning models are increasingly integrated into societally critical applications such as recidivism prediction and medical diagnosis, thanks to their superior predictive power.
"Why is 'Chicago' deceptive?" Towards Building Model-Driven Tutorials for Humans
To support human decision making with machine learning models, we often need to elucidate patterns embedded in the models that are unsalient, unknown, or counterintuitive to humans.
Preserving Causal Constraints in Counterfactual Explanations for Machine Learning Classifiers
For explanations of ML models in critical domains such as healthcare and finance, counterfactual examples are useful for an end-user only to the extent that perturbation of feature inputs is feasible in the real world.
What Gets Echoed? Understanding the ``Pointers'' in Explanations of Persuasive Arguments
Explanations are central to everyday life, and are a topic of growing interest in the AI community.
What Gets Echoed? Understanding the "Pointers" in Explanations of Persuasive Arguments
Explanations are central to everyday life, and are a topic of growing interest in the AI community.
Content Removal as a Moderation Strategy: Compliance and Other Outcomes in the ChangeMyView Community
Moderators of online communities often employ comment deletion as a tool.
Many Faces of Feature Importance: Comparing Built-in and Post-hoc Feature Importance in Text Classification
In this work, we systematically compare feature importance from built-in mechanisms in a model such as attention values and post-hoc methods that approximate model behavior such as LIME.
Gift Contagion in Online Groups: Evidence From Virtual Red Packets
Using data on 36 million online red packet gifts on a large social site in East Asia, we leverage a natural experimental design to identify the social contagion of gift giving in online groups.
Explaining Machine Learning Classifiers through Diverse Counterfactual Explanations
Post-hoc explanations of machine learning models are crucial for people to understand and act on algorithmic predictions.
Learning Fair Representations via an Adversarial Framework
Fairness has become a central issue for our research community as classification algorithms are adopted in societally critical domains such as recidivism prediction and loan approval.
No Permanent Friends or Enemies: Tracking Relationships between Nations from News
Understanding the dynamics of international politics is important yet challenging for civilians.
Measuring Online Debaters' Persuasive Skill from Text over Time
We build on a widely used model of skill in two-player games and augment it with linguistic features of a debater{'}s content.
Ask Not What AI Can Do, But What AI Should Do: Towards a Framework of Task Delegability
To obtain an empirical understanding of human preferences in different tasks, we build a dataset of 100 tasks from academic papers, popular media portrayal of AI, and everyday life, and administer a survey based on our proposed framework.
On Human Predictions with Explanations and Predictions of Machine Learning Models: A Case Study on Deception Detection
In this paper, we use deception detection as a testbed and investigate how we can harness explanations and predictions of machine learning models to improve human performance while retaining human agency.
LSTMs Exploit Linguistic Attributes of Data
While recurrent neural networks have found success in a variety of natural language processing applications, they are general models of sequential data.
"You are no Jack Kennedy": On Media Selection of Highlights from Presidential Debates
We first examine the effect of wording and propose a binary classification framework that controls for both the speaker and the debate situation.
Dynamic Entity Representations in Neural Language Models
Understanding a long document requires tracking how entities are introduced and evolve over time.
Neural Models for Documents with Metadata
Most real-world document collections involve various types of metadata, such as author, source, and date, and yet the most commonly-used approaches to modeling text corpora ignore this information.
Friendships, Rivalries, and Trysts: Characterizing Relations between Ideas in Texts
Combining two statistics --- cooccurrence within documents and prevalence correlation over time --- our approach reveals a number of different ways in which ideas can cooperate and compete.
Talk it up or play it down? (Un)expected correlations between (de-)emphasis and recurrence of discussion points in consequential U.S. economic policy meetings
In meetings where important decisions get made, what items receive more attention may influence the outcome.
Winning Arguments: Interaction Dynamics and Persuasion Strategies in Good-faith Online Discussions
Changing someone's opinion is arguably one of the most important challenges of social interaction.
All Who Wander: On the Prevalence and Characteristics of Multi-community Engagement
In this paper, we examine three aspects of multi-community engagement: the sequence of communities that users post to, the language that users employ in those communities, and the feedback that users receive, using longitudinal posting behavior on Reddit as our main data source, and DBLP for auxiliary experiments.
The effect of wording on message propagation: Topic- and author-controlled natural experiments on Twitter
Consider a person trying to spread an important message on a social network.
A Corpus of Sentence-level Revisions in Academic Writing: A Step towards Understanding Statement Strength in Communication
The strength with which a statement is made can have a significant impact on the audience.
