no code implementations • 1 Nov 2023 • Jonas F. Lotz, Elizabeth Salesky, Phillip Rust, Desmond Elliott
Pixel-based language models process text rendered as images, which allows them to handle any script, making them a promising approach to open vocabulary language modelling.
1 code implementation • 5 May 2023 • Wenyan Li, Jonas F. Lotz, Chen Qiu, Desmond Elliott
Image captioning models are typically trained by treating all samples equally, neglecting to account for mismatched or otherwise difficult data points.
1 code implementation • 14 Jul 2022 • Phillip Rust, Jonas F. Lotz, Emanuele Bugliarello, Elizabeth Salesky, Miryam de Lhoneux, Desmond Elliott
We pretrain the 86M parameter PIXEL model on the same English data as BERT and evaluate on syntactic and semantic tasks in typologically diverse languages, including various non-Latin scripts.
Ranked #1 on Named Entity Recognition (NER) on MasakhaNER