Search Results for author: Shane Steinert-Threlkeld

Found 17 papers, 11 papers with code

The Impact of Syntactic and Semantic Proximity on Machine Translation with Back-Translation

no code implementations26 Mar 2024 Nicolas Guerin, Shane Steinert-Threlkeld, Emmanuel Chemla

Unsupervised on-the-fly back-translation, in conjunction with multilingual pretraining, is the dominant method for unsupervised neural machine translation.

Translation Unsupervised Machine Translation

Embedding structure matters: Comparing methods to adapt multilingual vocabularies to new languages

1 code implementation9 Sep 2023 C. M. Downey, Terra Blevins, Nora Goldfine, Shane Steinert-Threlkeld

Pre-trained multilingual language models underpin a large portion of modern NLP tools outside of English.

Evaluating Transformer's Ability to Learn Mildly Context-Sensitive Languages

no code implementations2 Sep 2023 Shunjie Wang, Shane Steinert-Threlkeld

Despite the fact that Transformers perform well in NLP tasks, recent studies suggest that self-attention is theoretically limited in learning even some regular and context-free languages.

The Weighted Möbius Score: A Unified Framework for Feature Attribution

1 code implementation16 May 2023 Yifan Jiang, Shane Steinert-Threlkeld

Feature attribution aims to explain the reasoning behind a black-box model's prediction by identifying the impact of each feature on the prediction.

Sentiment Analysis

Probing for Understanding of English Verb Classes and Alternations in Large Pre-trained Language Models

no code implementations11 Sep 2022 David K. Yi, James V. Bruno, Jiayu Han, Peter Zukerman, Shane Steinert-Threlkeld

We investigate the extent to which verb alternation classes, as described by Levin (1993), are encoded in the embeddings of Large Pre-trained Language Models (PLMs) such as BERT, RoBERTa, ELECTRA, and DeBERTa using selectively constructed diagnostic classifiers for word and sentence-level prediction tasks.

Sentence

Learning to translate by learning to communicate

1 code implementation14 Jul 2022 C. M. Downey, Xuhui Zhou, Leo Z. Liu, Shane Steinert-Threlkeld

We formulate and test a technique to use Emergent Communication (EC) with a pre-trained multilingual model to improve on modern Unsupervised NMT systems, especially for low-resource languages.

Natural Language Understanding NMT

Language Models Use Monotonicity to Assess NPI Licensing

1 code implementation Findings (ACL) 2021 Jaap Jumelet, Milica Denić, Jakub Szymanik, Dieuwke Hupkes, Shane Steinert-Threlkeld

We investigate the semantic knowledge of language models (LMs), focusing on (1) whether these LMs create categories of linguistic environments based on their semantic monotonicity properties, and (2) whether these categories play a similar role in LMs as in human language understanding, using negative polarity item licensing as a case study.

Linguistic Acceptability

A multilabel approach to morphosyntactic probing

no code implementations Findings (EMNLP) 2021 Naomi Tachikawa Shapiro, Amandalynne Paullada, Shane Steinert-Threlkeld

We introduce a multilabel probing task to assess the morphosyntactic representations of word embeddings from multilingual language models.

Word Embeddings

A Masked Segmental Language Model for Unsupervised Natural Language Segmentation

1 code implementation NAACL (SIGMORPHON) 2022 C. M. Downey, Fei Xia, Gina-Anne Levow, Shane Steinert-Threlkeld

Segmentation remains an important preprocessing step both in languages where "words" or other important syntactic/semantic units (like morphemes) are not clearly delineated by white space, as well as when dealing with continuous speech data, where there is often no meaningful pause between words.

Language Modelling Segmentation

Bilingual alignment transfers to multilingual alignment for unsupervised parallel text mining

1 code implementation ACL 2022 Chih-chan Tien, Shane Steinert-Threlkeld

To study this theory, we design unsupervised models trained on unpaired sentences and single-pair supervised models trained on bitexts, both based on the unsupervised language model XLM-R with its parameters frozen.

Retrieval Sentence +1

Linguistically-Informed Transformations (LIT): A Method for Automatically Generating Contrast Sets

no code implementations EMNLP (BlackboxNLP) 2020 Chuanrong Li, Lin Shengshuo, Leo Z. Liu, Xinyi Wu, Xuhui Zhou, Shane Steinert-Threlkeld

Although large-scale pretrained language models, such as BERT and RoBERTa, have achieved superhuman performance on in-distribution test sets, their performance suffers on out-of-distribution test sets (e. g., on contrast sets).

On the Spontaneous Emergence of Discrete and Compositional Signals

1 code implementation ACL 2020 Nur Geffen Lan, Emmanuel Chemla, Shane Steinert-Threlkeld

We propose a general framework to study language emergence through signaling games with neural agents.

Paying Attention to Function Words

1 code implementation24 Sep 2019 Shane Steinert-Threlkeld

All natural languages exhibit a distinction between content words (like nouns and adjectives) and function words (like determiners, auxiliaries, prepositions).

reinforcement-learning Reinforcement Learning (RL)

Neural Models of the Psychosemantics of `Most'

1 code implementation WS 2019 Lewis O'Sullivan, Shane Steinert-Threlkeld

How are the meanings of linguistic expressions related to their use in concrete cognitive tasks?

Cannot find the paper you are looking for? You can Submit a new open access paper.