Parallel
Edit

Belebele

Introduced by Bandarkar et al. in The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variants

Belebele is a multiple-choice machine reading comprehension (MRC) dataset spanning 122 language variants. This dataset enables the evaluation of mono- and multi-lingual models in high-, medium-, and low-resource languages. Each question has four multiple-choice answers and is linked to a short passage from the FLORES-200 dataset. The human annotation procedure was carefully curated to create questions that discriminate between different levels of generalizable language comprehension and is reinforced by extensive quality checks. While all questions directly relate to the passage, the English dataset on its own proves difficult enough to challenge state-of-the-art language models. Being fully parallel, this dataset enables direct comparison of model performance across all languages. Belebele opens up new avenues for evaluating and analyzing the multilingual abilities of language models and NLP systems.

Homepage

Papers

Paper Code Results Date Stars

Dataset Loaders
Add Remove

No data loaders found. You can submit your data loader here.

Tasks

Similar Datasets

GPQA

GPQA
XStoryCloze

XStoryCloze
XCOPA

XCOPA
AfriQA

AfriQA

License

Modalities

Languages

Contact us on: hello@paperswithcode.com . Papers With Code is a free resource with all data licensed under CC-BY-SA.
Terms Data policy Cookies policy