Search Results for author:
Found 20 papers, 3 papers with code
MYTE: Morphology-Driven Byte Encoding for Better and Fairer Multilingual Language Modeling
A major consideration in multilingual language modeling is how to best represent languages with diverse vocabularies and scripts.
Comparing Hallucination Detection Metrics for Multilingual Generation
While many automatic hallucination detection techniques have been proposed for English texts, their effectiveness in multilingual contexts remains unexplored.
Breaking the Curse of Multilinguality with Cross-lingual Expert Language Models
Despite their popularity in non-English NLP, multilingual language models often underperform monolingual ones due to inter-language competition for model parameters.
Universal NER: A Gold-Standard Multilingual Named Entity Recognition Benchmark
We introduce Universal NER (UNER), an open, community-driven project to develop gold-standard NER benchmarks in many languages.
Ranked #1 on
Named Entity Recognition (NER)
on UNER v1 (Danish)
Detecting Pretraining Data from Large Language Models
Min-K% Prob can be applied without any knowledge about the pretraining corpus or any additional training, departing from previous detection methods that require training a reference model on data that is similar to the pretraining data.
Embedding structure matters: Comparing methods to adapt multilingual vocabularies to new languages
Pre-trained multilingual language models underpin a large portion of modern NLP tools outside of English.
BUFFET: Benchmarking Large Language Models for Few-shot Cross-lingual Transfer
Despite remarkable advancements in few-shot generalization in natural language processing, most models are developed and evaluated primarily in English.
Translate to Disambiguate: Zero-shot Multilingual Word Sense Disambiguation with Pretrained Language Models
To better understand this contrast, we present a new study investigating how well PLMs capture cross-lingual word sense with Contextual Word-Level Translation (C-WLT), an extension of word-level translation that prompts the model to translate a given word in context.
Demystifying Prompts in Language Models via Perplexity Estimation
Language models can be prompted to perform a wide variety of zero- and few-shot learning problems.
Prompting Language Models for Linguistic Structure
Although pretrained language models (PLMs) can be prompted to perform a wide range of language tasks, it remains an open question how much this ability comes from generalizable linguistic understanding versus surface-level lexical patterns.
Analyzing the Mono- and Cross-Lingual Pretraining Dynamics of Multilingual Language Models
The emergent cross-lingual transfer seen in multilingual pretrained models has sparked significant interest in studying their behavior.
Few-shot Mining of Naturally Occurring Inputs and Outputs
Creating labeled natural language training data is expensive and requires significant human effort.
Language Contamination Helps Explain the Cross-lingual Capabilities of English Pretrained Models
English pretrained language models, which make up the backbone of many modern NLP systems, require huge amounts of unlabeled training data.
FEWS: Large-Scale, Low-Shot Word Sense Disambiguation with the Dictionary
Current models for Word Sense Disambiguation (WSD) struggle to disambiguate rare senses, despite reaching human performance on global WSD metrics.
Moving Down the Long Tail of Word Sense Disambiguation with Gloss Informed Bi-encoders
A major obstacle in Word Sense Disambiguation (WSD) is that word senses are not uniformly distributed, causing existing models to generally perform poorly on senses that are either rare or unseen during training.
Ranked #9 on
Word Sense Disambiguation
on Supervised:
Moving Down the Long Tail of Word Sense Disambiguation with Gloss-Informed Biencoders
A major obstacle in Word Sense Disambiguation (WSD) is that word senses are not uniformly distributed, causing existing models to generally perform poorly on senses that are either rare or unseen during training.
Better Character Language Modeling Through Morphology
We incorporate morphological supervision into character language models (CLMs) via multitasking and show that this addition improves bits-per-character (BPC) performance across 24 languages, even when the morphology data and language modeling data are disjoint.
Deep RNNs Encode Soft Hierarchical Syntax
We present a set of experiments to demonstrate that deep recurrent neural networks (RNNs) learn internal representations that capture soft hierarchical notions of syntax from highly varied supervision.
Automatically Processing Tweets from Gang-Involved Youth: Towards Detecting Loss and Aggression
Violence is a serious problems for cities like Chicago and has been exacerbated by the use of social media by gang-involved youths for taunting rival gangs.
