Search Results for author:
Found 47 papers, 8 papers with code
Beyond calories: evaluating how tailored communication reduces emotional load in diet-coaching
However, previous work on apps evaluation only focused on dietary outcomes, ignoring users’ emotional state despite its influence on eating habits.
ReproGen: Proposal for a Shared Task on Reproducibility of Human Evaluations in NLG
Across NLP, a growing body of work is looking at the issue of reproducibility.
Shared Task on Evaluating Accuracy
We propose a shared task on methodologies and algorithms for evaluating the accuracy of generated texts, specifically summaries of basketball games produced from basketball box score and other game data.
Arabic NLG Language Functions
In this paper, we explain the challenges of the core grammar, provide a lexical resource, and implement the first language functions for the Arabic language.
Explaining Bayesian Networks in Natural Language: State of the Art and Challenges
In order to increase trust in the usage of Bayesian Networks and to cement their role as a model which can aid in critical decision making, the challenge of explainability must be faced.
Explaining Decision-Tree Predictions by Addressing Potential Conflicts between Predictions and Plausible Expectations
We offer an approach to explain Decision Tree (DT) predictions by addressing potential conflicts between aspects of these predictions and plausible expectations licensed by background information.
The ReproGen Shared Task on Reproducibility of Human Evaluations in NLG: Overview and Results
The NLP field has recently seen a substantial increase in work related to reproducibility of results, and more generally in recognition of the importance of having shared definitions and practices relating to evaluation.
Improving Factual Accuracy of Neural Table-to-Text Output by Addressing Input Problems in ToTTo
Neural Table-to-Text models tend to hallucinate, producing texts that contain factual errors.
Linguistically Communicating Uncertainty in Patient-Facing Risk Prediction Models
This paper addresses the unique challenges associated with uncertainty quantification in AI models when applied to patient-facing contexts within healthcare.
Explainable artificial intelligence
Explainable Artificial Intelligence (XAI)
+1
Textual Summarisation of Large Sets: Towards a General Approach
We are developing techniques to generate summary descriptions of sets of objects.
Ask the experts: sourcing high-quality datasets for nutritional counselling through Human-AI collaboration
We release HAI-coaching, the first expert-annotated nutrition counselling dataset containing ~2. 4K dietary struggles from crowd workers, and ~97K related supportive texts generated by ChatGPT.
Evaluation of Human-Understandability of Global Model Explanations using Decision Tree
A task based evaluation of mental models of these participants provide valuable feedback to enhance narrative global explanations.
Missing Information, Unresponsive Authors, Experimental Flaws: The Impossibility of Assessing the Reproducibility of Previous Human Evaluations in NLP
We report our efforts in identifying a set of previous human evaluations in NLP that would be suitable for a coordinated study examining what makes human evaluations in NLP more/less reproducible.
Consultation Checklists: Standardising the Human Evaluation of Medical Note Generation
Evaluating automatically generated text is generally hard due to the inherently subjective nature of many aspects of the output quality.
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions.
Comparing informativeness of an NLG chatbot vs graphical app in diet-information domain
Visual representation of data like charts and tables can be challenging to understand for readers.
User-Driven Research of Medical Note Generation Software
A growing body of work uses Natural Language Processing (NLP) methods to automatically generate medical notes from audio recordings of doctor-patient consultations.
Anno-MI: A Dataset of Expert-Annotated Counselling Dialogues
Research on natural language processing for counselling dialogue analysis has seen substantial development in recent years, but access to this area remains extremely limited due to the lack of publicly available expert-annotated therapy conversations.
Human Evaluation and Correlation with Automatic Metrics in Consultation Note Generation
In recent years, machine learning models have rapidly become better at generating clinical consultation notes; yet, there is little work on how to properly evaluate the generated consultation notes to understand the impact they may have on both the clinician using them and the patient's clinical safety.
Generation Challenges: Results of the Accuracy Evaluation Shared Task
The Shared Task on Evaluating Accuracy focused on techniques (both manual and automatic) for evaluating the factual accuracy of texts produced by neural NLG systems, in a sports-reporting domain.
Towards objectively evaluating the quality of generated medical summaries
We propose a method for evaluating the quality of generated text by asking evaluators to count facts, and computing precision, recall, f-score, and accuracy from the raw counts.
A preliminary study on evaluating Consultation Notes with Post-Editing
We time this and find that it is faster than writing the note from scratch.
A Systematic Review of Reproducibility Research in Natural Language Processing
Against the background of what has been termed a reproducibility crisis in science, the NLP field is becoming increasingly interested in, and conscientious about, the reproducibility of its results.
A Gold Standard Methodology for Evaluating Accuracy in Data-To-Text Systems
Most Natural Language Generation systems need to produce accurate texts.
SportSett:Basketball - A robust and maintainable dataset for Natural Language Generation
In this resource paper, we introduce the SportSett:Basketball database.
How are you? Introducing stress-based text tailoring
Can stress affect not only your life but also how you read and interpret a text?
Shared Task on Evaluating Accuracy in Natural Language Generation
We propose a shared task on methodologies and algorithms for evaluating the accuracy of generated texts.
FitChat: Conversational Artificial Intelligence Interventions for Encouraging Physical Activity in Older Adults
For this reason, chatbots have recently been seen in healthcare delivering digital interventions through free text or choice selection.
Natural Language Generation Challenges for Explainable AI
Good quality explanations of artificial intelligence (XAI) reasoning must be written (and evaluated) for an explanatory purpose, targeted towards their readers, have a good narrative and causal structure, and highlight where uncertainty and data quality affect the AI output.
Comprehension Driven Document Planning in Natural Language Generation Systems
This paper proposes an approach to NLG system design which focuses on generating output text which can be more easily processed by the reader.
Generating Summaries of Sets of Consumer Products: Learning from Experiments
We explored the task of creating a textual summary describing a large set of objects characterised by a small number of features using an e-commerce dataset.
Meteorologists and Students: A resource for language grounding of geographical descriptors
We present a data resource which can be useful for research purposes on language grounding tasks in the context of geographical referring expression generation.
A Structured Review of the Validity of BLEU
The BLEU metric has been widely used in NLP for over 15 years to evaluate NLP systems, especially in machine translation and natural language generation.
Making effective use of healthcare data using data-to-text technology
Financial reports are produced to assess healthcare organizations on some key performance indicators to steer their healthcare delivery.
Textually Summarising Incomplete Data
Many data-to-text NLG systems work with data sets which are incomplete, ie some of the data is missing.
A Commercial Perspective on Reference
I briefly describe some of the commercial work which XXX is doing in referring expression algorithms, and highlight differences between what is commercially important (at least to XXX) and the NLG research literature.
An Empirical Approach for Modeling Fuzzy Geographical Descriptors
We present a novel heuristic approach that defines fuzzy geographical descriptors using data gathered from a survey with human subjects.
NLG vs. Templates
One of the most important questions in applied NLG is what benefits (or `value-added', in business-speak) NLG technology offers over template-based approaches.
