Search Results for author:
Found 18 papers, 9 papers with code
Revenge of the Fallen? Recurrent Models Match Transformers at Predicting Human Language Comprehension Metrics
Transformers have supplanted Recurrent Neural Networks as the dominant architecture for both natural language processing tasks and, despite criticisms of cognitive implausibility, for modelling the effect of predictability on online human language comprehension.
A Bit of a Problem: Measurement Disparities in Dataset Sizes Across Languages
We release a tool to obtain byte premiums for any two languages, enabling comparisons of dataset sizes across languages for more equitable multilingual model development and data practices.
When Is Multilinguality a Curse? Language Modeling for 250 High- and Low-Resource Languages
However, concrete evidence for the effects of multilinguality on language modeling performance in individual languages remains scarce.
Structural Priming Demonstrates Abstract Grammatical Representations in Multilingual Language Models
We measure crosslingual structural priming in large language models, comparing model behavior to human experimental results from eight crosslingual experiments covering six languages, and four monolingual structural priming experiments in three non-English languages.
Crosslingual Structural Priming and the Pre-Training Dynamics of Bilingual Language Models
Do multilingual language models share abstract grammatical representations across languages, and if so, when do these develop?
Characterizing Learning Curves During Language Model Pre-Training: Learning, Forgetting, and Stability
We quantify the final surprisal, within-run variability, age of acquisition, forgettability, and cross-run variability of learning curves for individual tokens in context.
Emergent inabilities? Inverse scaling over the course of pretraining
Does inverse scaling only occur as a function of model parameter size, or can it also occur over the course of training?
Language Model Behavior: A Comprehensive Survey
Transformer language models have received widespread public attention, yet their generated text is often surprising even to NLP researchers.
Can Peanuts Fall in Love with Distributional Semantics?
Context changes expectations about upcoming words - following a story involving an anthropomorphic peanut, comprehenders expect the sentence the peanut was in love more than the peanut was salted, as indexed by N400 amplitude (Nieuwland & van Berkum, 2006).
Rarely a problem? Language models exhibit inverse scaling in their predictions following few-type quantifiers
How well do language models deal with quantification?
Collateral facilitation in humans and language models
Are the predictions of humans and language models affected by similar things?
Do language models make human-like predictions about the coreferents of Italian anaphoric zero pronouns?
Some languages allow arguments to be omitted in certain contexts.
The Geometry of Multilingual Language Model Representations
The subspace means differ along language-sensitive axes that are relatively stable throughout middle layers, and these axes encode information such as token vocabularies.
Word Acquisition in Neural Language Models
We investigate how neural language models acquire individual words during training, extracting learning curves and ages of acquisition for over 600 words on the MacArthur-Bates Communicative Development Inventory (Fenson et al., 2007).
So Cloze yet so Far: N400 Amplitude is Better Predicted by Distributional Information than Human Predictability Judgements
In this study, we investigate whether the linguistic predictions of computational language models or humans better reflect the way in which natural language stimuli modulate the amplitude of the N400.
Different kinds of cognitive plausibility: why are transformers better than RNNs at predicting N400 amplitude?
Despite being designed for performance rather than cognitive plausibility, transformer language models have been found to be better at predicting metrics used to assess human language comprehension than language models with other architectures, such as recurrent neural networks.
How well does surprisal explain N400 amplitude under different experimental conditions?
We investigate the extent to which word surprisal can be used to predict a neural measure of human language processing difficulty - the N400.
