no code implementations • WMT (EMNLP) 2020 • Nitika Mathur, Johnny Wei, Markus Freitag, Qingsong Ma, Ondřej Bojar
Participants were asked to score the outputs of the translation systems competing in the WMT20 News Translation Task with automatic metrics.
no code implementations • WMT (EMNLP) 2021 • Markus Freitag, Ricardo Rei, Nitika Mathur, Chi-kiu Lo, Craig Stewart, George Foster, Alon Lavie, Ondřej Bojar
Contrary to previous years’ editions, this year we acquired our own human ratings based on expert-based human evaluation via Multidimensional Quality Metrics (MQM).
1 code implementation • ACL 2020 • Nitika Mathur, Timothy Baldwin, Trevor Cohn
Automatic metrics are fundamental for the development and evaluation of machine translation systems.
1 code implementation • ACL 2019 • Nitika Mathur, Timothy Baldwin, Trevor Cohn
Accurate, automatic evaluation of machine translation is critical for system tuning, and evaluating progress in the field.
no code implementations • ALTA 2018 • Nitika Mathur, Timothy Baldwin, Trevor Cohn
In this paper we show that the quality control mechanism is overly conservative, which increases the time and expense of the evaluation.
no code implementations • EMNLP 2017 • Nitika Mathur, Timothy Baldwin, Trevor Cohn
Manual data annotation is a vital component of NLP research.