Search Results for author:
Found 12 papers, 4 papers with code
Can Transformers Process Recursive Nested Constructions, Like Humans?
A recent study evaluated recursive processing in recurrent neural language models (RNN-LMs) and showed that such models perform below chance level on embedded dependencies within nested constructions – a prototypical example of recursion in natural language.
Metric-Learning Encoding Models Identify Processing Profiles of Linguistic Features in BERT's Representations
Together, this demonstrates the utility of Metric-Learning Encoding Methods for studying how linguistic features are neurally encoded in language models and the advantage of MLEMs over traditional methods.
Language acquisition: do children and language models follow similar learning stages?
To investigate this, we here compare the learning trajectories of deep language models to those of children.
Probing Brain Context-Sensitivity with Masked-Attention Generation
Two fundamental questions in neurolinguistics concerns the brain regions that integrate information beyond the lexical level, and the size of their window of integration.
Information-Restricted Neural Language Models Reveal Different Brain Regions' Sensitivity to Semantics, Syntax and Context
A fundamental question in neurolinguistics concerns the brain regions involved in syntactic and semantic processing during speech comprehension, both at the lexical (word processing) and supra-lexical levels (sentence and discourse processing).
Neural Language Models are not Born Equal to Fit Brain Data, but Training Helps
Neural Language Models (NLMs) have made tremendous advances during the last years, achieving impressive performance on various linguistic tasks.
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
BIG-bench focuses on tasks that are believed to be beyond the capabilities of current language models.
Causal Transformers Perform Below Chance on Recursive Nested Constructions, Unlike Humans
A recent study evaluated recursive processing in recurrent neural language models (RNN-LMs) and showed that such models perform below chance level on embedded dependencies within nested constructions -- a prototypical example of recursion in natural language.
Can RNNs learn Recursive Nested Subject-Verb Agreements?
Finally, probing the internal states of the model during the processing of sentences with nested tree structures, we found a complex encoding of grammatical agreement information (e. g. grammatical number), in which all the information for multiple words nouns was carried by a single unit.
Mechanisms for Handling Nested Dependencies in Neural-Network Language Models and Humans
We studied whether a modern artificial neural network trained with "deep learning" methods mimics a central aspect of human sentence processing, namely the storing of grammatical number and gender information in working memory and its use in long-distance agreement (e. g., capturing the correct number agreement between subject and verb when they are separated by other phrases).
The emergence of number and syntax units in LSTM language models
Importantly, the behaviour of these units is partially controlled by other units independently shown to track syntactic structure.
Metric Learning for Phoneme Perception
This study presents a new framework for learning a metric function for perceptual distances among pairs of phonemes.
