GlobalBench: A Benchmark for Global Progress in Natural Language Processing

no code implementations • 24 May 2023 • Yueqi Song, Catherine Cui, Simran Khanuja, PengFei Liu, Fahim Faisal, Alissa Ostapenko, Genta Indra Winata, Alham Fikri Aji, Samuel Cahyawijaya, Yulia Tsvetkov, Antonios Anastasopoulos, Graham Neubig

Despite the major advances in NLP, significant disparities in NLP system performance across languages still exist.

Speaker Information Can Guide Models to Better Inductive Biases: A Case Study On Predicting Code-Switching

1 code implementation • ACL 2022 • Alissa Ostapenko, Shuly Wintner, Melinda Fricke, Yulia Tsvetkov

Natural language processing (NLP) models trained on people-generated data can be unreliable because, without any constraints, they can learn from spurious correlations that are not relevant to the task.

Rethinking End-to-End Evaluation of Decomposable Tasks: A Case Study on Spoken Language Understanding

no code implementations • 29 Jun 2021 • Siddhant Arora, Alissa Ostapenko, Vijay Viswanathan, Siddharth Dalmia, Florian Metze, Shinji Watanabe, Alan W Black

Our splits identify performance gaps up to 10% between end-to-end systems that were within 1% of each other on the original test sets.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +3