ALIGN-MLM: Word Embedding Alignment is Crucial for Multilingual Pre-training

1 code implementation • 15 Nov 2022 • Henry Tang, Ameet Deshpande, Karthik Narasimhan

In particular, ALIGN-MLM outperforms XLM and MLM by 35 and 30 F1 points on POS-tagging for transfer between languages that differ both in their script and word order (left-to-right v. s.

POS POS Tagging +2