Search Results for author:
Found 7 papers, 2 papers with code
A General Model for Aggregating Annotations Across Simple, Complex, and Multi-Object Annotation Tasks
Beyond investigating these research questions above, we discuss the foundational concept of annotation complexity, present a new aggregation model as a bridge between traditional models and our own, and contribute a new semi-supervised learning method for complex label aggregation that outperforms prior work.
Measuring Annotator Agreement Generally across Complex Structured, Multi-object, and Free-text Annotation Tasks
When annotators label data, a key metric for quality assurance is inter-annotator agreement (IAA): the extent to which annotators agree on their labels.
Local versus Global Strategies in Social Query Expansion
Link sharing in social media can be seen as a collaboratively retrieved set of documents for a query or topic expressed by a hashtag.
Scalable Knowledge Graph Construction from Twitter
We describe a knowledge graph derived from Twitter data with the goal of discovering relationships between people, links, and topics.
How Many Workers to Ask? Adaptive Exploration for Collecting High Quality Labels
In this paper we investigate how to devise better stopping rules given such quality scores.
A Data Management Approach for Dataset Selection Using Human Computation
As the number of applications that use machine learning algorithms increases, the need for labeled data useful for training such algorithms intensifies.
Adaptive Crowdsourcing Algorithms for the Bandit Survey Problem
This model is related to, but technically different from the well-known multi-armed bandit problem.
