Search Results for author:
Found 10 papers, 1 papers with code
Identifying Annotator Bias: A new IRT-based method for bias identification
Our interpretation of IRT offers an original bias identification method that can be used to compare annotators{'} bias and characterise annotation disagreement.
Agreement is overrated: A plea for correlation to assess human evaluation reliability
Following Sampson and Babarczy (2008), Lommel et al. (2014), Joshi et al. (2016) and Amidei et al. (2018b), such phenomena can be explained in terms of irreducible human language variability.
The use of rating and Likert scales in Natural Language Generation human evaluation tasks: A review and some recommendations
Rating and Likert scales are widely used in evaluation experiments to measure the quality of Natural Language Generation (NLG) systems.
Evaluation methodologies in Automatic Question Generation 2013-2018
In the last few years Automatic Question Generation (AQG) has attracted increasing interest.
Rethinking the Agreement in Human Evaluation Tasks
For this reason, we believe that annotation schemes for natural language generation tasks that are aimed at evaluating language quality need to be treated with great care.
Search Personalization with Embeddings
Recent research has shown that the performance of search personalization depends on the richness of user profiles which normally represent the user's topical interests.
Adverse Drug Reaction Classification With Deep Neural Networks
On the Twitter dataset, all the NN architectures perform similarly.
