TASK	DATASET	MODEL	METRIC NAME	METRIC VALUE	GLOBAL RANK
Cross-Lingual NER	UNER v1 (Cebuano)	UNER XML-R (all)	F1 (micro)	69.6	# 1
Named Entity Recognition (NER)	UNER v1 (Chinese)	UNER XML-R	F1 (micro)	89.50	# 1
Cross-Lingual NER	UNER v1 (Chinese)	UNER XML-R (all)	F1 (micro)	88.2	# 1
Named Entity Recognition (NER)	UNER v1 (Chinese Simplified)	UNER XML-R	F1 (micro)	89.40	# 1
Cross-Lingual NER	UNER v1 (Chinese Simplified)	UNER XML-R (all)	F1 (micro)	87.7	# 1
Named Entity Recognition (NER)	UNER v1 (Croatian)	UNER XML-R	F1 (micro)	93.60	# 1
Cross-Lingual NER	UNER v1 (Croatian)	UNER XML-R (all)	F1 (micro)	90.9	# 1
Cross-Lingual NER	UNER v1 (Danish)	UNER XML-R (all)	F1 (micro)	83.0	# 1
Named Entity Recognition (NER)	UNER v1 (Danish)	UNER XML-R	F1 (micro)	82.70	# 1
Cross-Lingual NER	UNER v1 (English)	UNER XML-R (all)	F1 (micro)	82.8	# 1
Named Entity Recognition (NER)	UNER v1 (English)	UNER XML-R	F1 (micro)	86.00	# 1
Cross-Lingual NER	UNER v1 (Portuguese)	UNER XML-R (all)	F1 (micro)	82.3	# 1
Named Entity Recognition (NER)	UNER v1 (Portuguese)	UNER XML-R	F1 (micro)	90.4	# 1
Cross-Lingual NER	UNER v1 - PUD (Chinese)	UNER XML-R (all)	F1 (micro)	86.0	# 1
Named Entity Recognition (NER)	UNER v1 - PUD (Chinese)	UNER XML-R	F1 (micro)	87.10	# 1
Cross-Lingual NER	UNER v1 - PUD (English)	UNER XML-R (all)	F1 (micro)	79.5	# 1
Named Entity Recognition (NER)	UNER v1 - PUD (English)	UNER XML-R	F1 (micro)	80.10	# 1
Cross-Lingual NER	UNER v1 - PUD (German)	UNER XML-R (all)	F1 (micro)	78.9	# 1
Named Entity Recognition (NER)	UNER v1 - PUD (Portuguese)	UNER XML-R	F1 (micro)	88.80	# 1
Cross-Lingual NER	UNER v1 - PUD (Portuguese)	UNER XML-R (all)	F1 (micro)	85.1	# 1
Cross-Lingual NER	UNER v1 - PUD (Russian)	UNER XML-R (all)	F1 (micro)	70.6	# 1
Cross-Lingual NER	UNER v1 - PUD (Swedish)	UNER XML-R (all)	F1 (micro)	85.3	# 1
Named Entity Recognition (NER)	UNER v1 - PUD (Swedish)	UNER XML-R	F1 (micro)	82.20	# 1
Named Entity Recognition (NER)	UNER v1 (Serbian)	UNER XML-R	F1 (micro)	94.70	# 1
Cross-Lingual NER	UNER v1 (Serbian)	UNER XML-R (all)	F1 (micro)	95.2	# 1
Named Entity Recognition (NER)	UNER v1 (Slovak)	UNER XML-R	F1 (micro)	85.50	# 1
Cross-Lingual NER	UNER v1 (Slovak)	UNER XML-R (all)	F1 (micro)	81.6	# 1
Cross-Lingual NER	UNER v1 (Swedish)	UNER XML-R (all)	F1 (micro)	88.2	# 1
Named Entity Recognition (NER)	UNER v1 (Swedish)	UNER XML-R	F1 (micro)	88.30	# 1
Cross-Lingual NER	UNER v1 (Tagalog T)	UNER XML-R (all)	F1 (micro)	91.3	# 1
Cross-Lingual NER	UNER v1 (Tagalog U)	UNER XML-R (all)	F1 (micro)	63.8	# 1

Badge	Markdown
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/universal-ner-a-gold-standard-multilingual-1/cross-lingual-ner-on-uner-v1-cebuano)](https://paperswithcode.com/sota/cross-lingual-ner-on-uner-v1-cebuano?p=universal-ner-a-gold-standard-multilingual-1)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/universal-ner-a-gold-standard-multilingual-1/named-entity-recognition-ner-on-uner-v1-7)](https://paperswithcode.com/sota/named-entity-recognition-ner-on-uner-v1-7?p=universal-ner-a-gold-standard-multilingual-1)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/universal-ner-a-gold-standard-multilingual-1/cross-lingual-ner-on-uner-v1-chinese)](https://paperswithcode.com/sota/cross-lingual-ner-on-uner-v1-chinese?p=universal-ner-a-gold-standard-multilingual-1)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/universal-ner-a-gold-standard-multilingual-1/named-entity-recognition-ner-on-uner-v1-8)](https://paperswithcode.com/sota/named-entity-recognition-ner-on-uner-v1-8?p=universal-ner-a-gold-standard-multilingual-1)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/universal-ner-a-gold-standard-multilingual-1/cross-lingual-ner-on-uner-v1-chinese-1)](https://paperswithcode.com/sota/cross-lingual-ner-on-uner-v1-chinese-1?p=universal-ner-a-gold-standard-multilingual-1)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/universal-ner-a-gold-standard-multilingual-1/named-entity-recognition-ner-on-uner-v1-2)](https://paperswithcode.com/sota/named-entity-recognition-ner-on-uner-v1-2?p=universal-ner-a-gold-standard-multilingual-1)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/universal-ner-a-gold-standard-multilingual-1/cross-lingual-ner-on-uner-v1-croatian)](https://paperswithcode.com/sota/cross-lingual-ner-on-uner-v1-croatian?p=universal-ner-a-gold-standard-multilingual-1)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/universal-ner-a-gold-standard-multilingual-1/cross-lingual-ner-on-uner-v1-danish)](https://paperswithcode.com/sota/cross-lingual-ner-on-uner-v1-danish?p=universal-ner-a-gold-standard-multilingual-1)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/universal-ner-a-gold-standard-multilingual-1/named-entity-recognition-ner-on-uner-v1)](https://paperswithcode.com/sota/named-entity-recognition-ner-on-uner-v1?p=universal-ner-a-gold-standard-multilingual-1)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/universal-ner-a-gold-standard-multilingual-1/cross-lingual-ner-on-uner-v1-english)](https://paperswithcode.com/sota/cross-lingual-ner-on-uner-v1-english?p=universal-ner-a-gold-standard-multilingual-1)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/universal-ner-a-gold-standard-multilingual-1/named-entity-recognition-ner-on-uner-v1-1)](https://paperswithcode.com/sota/named-entity-recognition-ner-on-uner-v1-1?p=universal-ner-a-gold-standard-multilingual-1)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/universal-ner-a-gold-standard-multilingual-1/cross-lingual-ner-on-uner-v1-portuguese)](https://paperswithcode.com/sota/cross-lingual-ner-on-uner-v1-portuguese?p=universal-ner-a-gold-standard-multilingual-1)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/universal-ner-a-gold-standard-multilingual-1/named-entity-recognition-ner-on-uner-v1-3)](https://paperswithcode.com/sota/named-entity-recognition-ner-on-uner-v1-3?p=universal-ner-a-gold-standard-multilingual-1)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/universal-ner-a-gold-standard-multilingual-1/cross-lingual-ner-on-uner-v1-pud-chinese)](https://paperswithcode.com/sota/cross-lingual-ner-on-uner-v1-pud-chinese?p=universal-ner-a-gold-standard-multilingual-1)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/universal-ner-a-gold-standard-multilingual-1/named-entity-recognition-ner-on-uner-v1-pud-3)](https://paperswithcode.com/sota/named-entity-recognition-ner-on-uner-v1-pud-3?p=universal-ner-a-gold-standard-multilingual-1)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/universal-ner-a-gold-standard-multilingual-1/cross-lingual-ner-on-uner-v1-pud-english)](https://paperswithcode.com/sota/cross-lingual-ner-on-uner-v1-pud-english?p=universal-ner-a-gold-standard-multilingual-1)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/universal-ner-a-gold-standard-multilingual-1/named-entity-recognition-ner-on-uner-v1-pud)](https://paperswithcode.com/sota/named-entity-recognition-ner-on-uner-v1-pud?p=universal-ner-a-gold-standard-multilingual-1)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/universal-ner-a-gold-standard-multilingual-1/cross-lingual-ner-on-uner-v1-pud-german)](https://paperswithcode.com/sota/cross-lingual-ner-on-uner-v1-pud-german?p=universal-ner-a-gold-standard-multilingual-1)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/universal-ner-a-gold-standard-multilingual-1/named-entity-recognition-ner-on-uner-v1-pud-1)](https://paperswithcode.com/sota/named-entity-recognition-ner-on-uner-v1-pud-1?p=universal-ner-a-gold-standard-multilingual-1)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/universal-ner-a-gold-standard-multilingual-1/cross-lingual-ner-on-uner-v1-pud-portuguese)](https://paperswithcode.com/sota/cross-lingual-ner-on-uner-v1-pud-portuguese?p=universal-ner-a-gold-standard-multilingual-1)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/universal-ner-a-gold-standard-multilingual-1/cross-lingual-ner-on-uner-v1-pud-russian)](https://paperswithcode.com/sota/cross-lingual-ner-on-uner-v1-pud-russian?p=universal-ner-a-gold-standard-multilingual-1)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/universal-ner-a-gold-standard-multilingual-1/cross-lingual-ner-on-uner-v1-pud-swedish)](https://paperswithcode.com/sota/cross-lingual-ner-on-uner-v1-pud-swedish?p=universal-ner-a-gold-standard-multilingual-1)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/universal-ner-a-gold-standard-multilingual-1/named-entity-recognition-ner-on-uner-v1-pud-2)](https://paperswithcode.com/sota/named-entity-recognition-ner-on-uner-v1-pud-2?p=universal-ner-a-gold-standard-multilingual-1)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/universal-ner-a-gold-standard-multilingual-1/named-entity-recognition-ner-on-uner-v1-5)](https://paperswithcode.com/sota/named-entity-recognition-ner-on-uner-v1-5?p=universal-ner-a-gold-standard-multilingual-1)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/universal-ner-a-gold-standard-multilingual-1/cross-lingual-ner-on-uner-v1-serbian)](https://paperswithcode.com/sota/cross-lingual-ner-on-uner-v1-serbian?p=universal-ner-a-gold-standard-multilingual-1)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/universal-ner-a-gold-standard-multilingual-1/named-entity-recognition-ner-on-uner-v1-4)](https://paperswithcode.com/sota/named-entity-recognition-ner-on-uner-v1-4?p=universal-ner-a-gold-standard-multilingual-1)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/universal-ner-a-gold-standard-multilingual-1/cross-lingual-ner-on-uner-v1-slovak)](https://paperswithcode.com/sota/cross-lingual-ner-on-uner-v1-slovak?p=universal-ner-a-gold-standard-multilingual-1)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/universal-ner-a-gold-standard-multilingual-1/cross-lingual-ner-on-uner-v1-swedish)](https://paperswithcode.com/sota/cross-lingual-ner-on-uner-v1-swedish?p=universal-ner-a-gold-standard-multilingual-1)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/universal-ner-a-gold-standard-multilingual-1/named-entity-recognition-ner-on-uner-v1-6)](https://paperswithcode.com/sota/named-entity-recognition-ner-on-uner-v1-6?p=universal-ner-a-gold-standard-multilingual-1)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/universal-ner-a-gold-standard-multilingual-1/cross-lingual-ner-on-uner-v1-tagalog-t)](https://paperswithcode.com/sota/cross-lingual-ner-on-uner-v1-tagalog-t?p=universal-ner-a-gold-standard-multilingual-1)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/universal-ner-a-gold-standard-multilingual-1/cross-lingual-ner-on-uner-v1-tagalog-u)](https://paperswithcode.com/sota/cross-lingual-ner-on-uner-v1-tagalog-u?p=universal-ner-a-gold-standard-multilingual-1)`

Universal NER: A Gold-Standard Multilingual Named Entity Recognition Benchmark

arXiv 2023 · Stephen Mayhew, Terra Blevins, Shuheng Liu, Marek Šuppa, Hila Gonen, Joseph Marvin Imperial, Börje F. Karlsson, Peiqin Lin, Nikola Ljubešić, LJ Miranda, Barbara Plank, Arij Riabi, Yuval Pinter ·

We introduce Universal NER (UNER), an open, community-driven project to develop gold-standard NER benchmarks in many languages. The overarching goal of UNER is to provide high-quality, cross-lingually consistent annotations to facilitate and standardize multilingual NER research. UNER v1 contains 18 datasets annotated with named entities in a cross-lingual consistent schema across 12 diverse languages. In this paper, we detail the dataset creation and composition of UNER; we also provide initial modeling baselines on both in-language and cross-lingual learning settings. We release the data, code, and fitted models to the public.

PDF Abstract arXiv 2023 PDF

Code

Add Remove Mark official

UniversalNER/uner_code

Tasks

Add Remove

Cross-Lingual NER

Multilingual Named Entity Recognition

named-entity-recognition

Named Entity Recognition

Named Entity Recognition (NER)

NER

Datasets

Introduced in the Paper:

UNER v1

Used in the Paper:

CoNLL 2003

Universal Dependencies

CoNLL 2002 WikiANN MasakhaNER

Results from the Paper

Edit

Ranked #1 on Named Entity Recognition (NER) on UNER v1 (Danish)

Get a GitHub badge

Task	Dataset	Model	Metric Name	Metric Value	Global Rank	Benchmark
Cross-Lingual NER	UNER v1 (Cebuano)	UNER XML-R (all)	F1 (micro)	69.6	# 1	Compare
Named Entity Recognition (NER)	UNER v1 (Chinese)	UNER XML-R	F1 (micro)	89.50	# 1	Compare
Cross-Lingual NER	UNER v1 (Chinese)	UNER XML-R (all)	F1 (micro)	88.2	# 1	Compare
Named Entity Recognition (NER)	UNER v1 (Chinese Simplified)	UNER XML-R	F1 (micro)	89.40	# 1	Compare
Cross-Lingual NER	UNER v1 (Chinese Simplified)	UNER XML-R (all)	F1 (micro)	87.7	# 1	Compare
Named Entity Recognition (NER)	UNER v1 (Croatian)	UNER XML-R	F1 (micro)	93.60	# 1	Compare
Cross-Lingual NER	UNER v1 (Croatian)	UNER XML-R (all)	F1 (micro)	90.9	# 1	Compare
Cross-Lingual NER	UNER v1 (Danish)	UNER XML-R (all)	F1 (micro)	83.0	# 1	Compare
Named Entity Recognition (NER)	UNER v1 (Danish)	UNER XML-R	F1 (micro)	82.70	# 1	Compare
Cross-Lingual NER	UNER v1 (English)	UNER XML-R (all)	F1 (micro)	82.8	# 1	Compare
Named Entity Recognition (NER)	UNER v1 (English)	UNER XML-R	F1 (micro)	86.00	# 1	Compare
Cross-Lingual NER	UNER v1 (Portuguese)	UNER XML-R (all)	F1 (micro)	82.3	# 1	Compare
Named Entity Recognition (NER)	UNER v1 (Portuguese)	UNER XML-R	F1 (micro)	90.4	# 1	Compare
Cross-Lingual NER	UNER v1 - PUD (Chinese)	UNER XML-R (all)	F1 (micro)	86.0	# 1	Compare
Named Entity Recognition (NER)	UNER v1 - PUD (Chinese)	UNER XML-R	F1 (micro)	87.10	# 1	Compare
Cross-Lingual NER	UNER v1 - PUD (English)	UNER XML-R (all)	F1 (micro)	79.5	# 1	Compare
Named Entity Recognition (NER)	UNER v1 - PUD (English)	UNER XML-R	F1 (micro)	80.10	# 1	Compare
Cross-Lingual NER	UNER v1 - PUD (German)	UNER XML-R (all)	F1 (micro)	78.9	# 1	Compare
Named Entity Recognition (NER)	UNER v1 - PUD (Portuguese)	UNER XML-R	F1 (micro)	88.80	# 1	Compare
Cross-Lingual NER	UNER v1 - PUD (Portuguese)	UNER XML-R (all)	F1 (micro)	85.1	# 1	Compare
Cross-Lingual NER	UNER v1 - PUD (Russian)	UNER XML-R (all)	F1 (micro)	70.6	# 1	Compare
Cross-Lingual NER	UNER v1 - PUD (Swedish)	UNER XML-R (all)	F1 (micro)	85.3	# 1	Compare
Named Entity Recognition (NER)	UNER v1 - PUD (Swedish)	UNER XML-R	F1 (micro)	82.20	# 1	Compare
Named Entity Recognition (NER)	UNER v1 (Serbian)	UNER XML-R	F1 (micro)	94.70	# 1	Compare
Cross-Lingual NER	UNER v1 (Serbian)	UNER XML-R (all)	F1 (micro)	95.2	# 1	Compare
Named Entity Recognition (NER)	UNER v1 (Slovak)	UNER XML-R	F1 (micro)	85.50	# 1	Compare
Cross-Lingual NER	UNER v1 (Slovak)	UNER XML-R (all)	F1 (micro)	81.6	# 1	Compare
Cross-Lingual NER	UNER v1 (Swedish)	UNER XML-R (all)	F1 (micro)	88.2	# 1	Compare
Named Entity Recognition (NER)	UNER v1 (Swedish)	UNER XML-R	F1 (micro)	88.30	# 1	Compare
Cross-Lingual NER	UNER v1 (Tagalog T)	UNER XML-R (all)	F1 (micro)	91.3	# 1	Compare
Cross-Lingual NER	UNER v1 (Tagalog U)	UNER XML-R (all)	F1 (micro)	63.8	# 1	Compare

Methods

Add Remove

No methods listed for this paper. Add relevant methods here

Edit Social Preview

Universal NER: A Gold-Standard Multilingual Named Entity Recognition Benchmark

Code Edit Add Remove Mark official

Tasks Edit Add Remove

Datasets Edit

Results from the Paper Edit

Methods Edit Add Remove

Code

Add Remove Mark official

Tasks

Add Remove

Datasets

Results from the Paper

Edit

Methods

Add Remove