1 code implementation • 18 Jan 2021 • Sebastian Hofstätter, Aldo Lipani, Sophia Althammer, Markus Zlabinger, Allan Hanbury
In this work we analyze position bias on datasets, the contextualized representations, and their effect on retrieval results.
2 code implementations • Findings of the Association for Computational Linguistics 2020 • Markus Zlabinger, Marta Sabou, Sebastian Hofst{\"a}tter, Allan Hanbury
Obtaining such a corpus from crowdworkers, however, has been shown to be ineffective since (i) workers usually lack domain-specific expertise to conduct the task with sufficient quality, and (ii) the standard approach of annotating entire abstracts of trial reports as one task-instance (i. e. HIT) leads to an uneven distribution in task effort.
1 code implementation • 12 Aug 2020 • Sebastian Hofstätter, Markus Zlabinger, Mete Sertkan, Michael Schröder, Allan Hanbury
We extend the ranked retrieval annotations of the Deep Learning track of TREC 2019 with passage and word level graded relevance annotations for all relevant documents.
1 code implementation • 17 May 2020 • Markus Zlabinger, Marta Sabou, Sebastian Hofstätter, Mete Sertkan, Allan Hanbury
of 0. 68 to experts in DEXA vs. 0. 40 in CONTROL); (ii) already three per majority voting aggregated annotations of the DEXA approach reach substantial agreements to experts of 0. 78/0. 75/0. 69 for P/I/O (in CONTROL 0. 73/0. 58/0. 46).
1 code implementation • 4 Feb 2020 • Sebastian Hofstätter, Markus Zlabinger, Allan Hanbury
In addition, to gain insight into TK, we perform a clustered query analysis of TK's results, highlighting its strengths and weaknesses on queries with different types of information need and we show how to interpret the cause of ranking differences of two documents by comparing their internal scores.
no code implementations • 15 Jan 2020 • Markus Zlabinger, Sebastian Hofstätter, Navid Rekabsaz, Allan Hanbury
While existing disease-symptom relationship extraction methods are used as the foundation in the various medical tasks, no collection is available to systematically evaluate the performance of such methods.
1 code implementation • 10 Dec 2019 • Sebastian Hofstätter, Markus Zlabinger, Allan Hanbury
In this paper we look beyond metrics-based evaluation of Information Retrieval systems, to explore the reasons behind ranking results.
1 code implementation • 3 Dec 2019 • Sebastian Hofstätter, Markus Zlabinger, Allan Hanbury
The usage of neural network models puts multiple objectives in conflict with each other: Ideally we would like to create a neural model that is effective, efficient, and interpretable at the same time.