Search Results for author:
Found 17 papers, 2 papers with code
Word Order Does Matter and Shuffled Language Models Know It
Recent studies have shown that language models pretrained and/or fine-tuned on randomly permuted sentences exhibit competitive performance on GLUE, putting into question the importance of word order information.
Word Order Does Matter (And Shuffled Language Models Know It)
Recent studies have shown that language models pretrained and/or fine-tuned on randomly permuted sentences exhibit competitive performance on GLUE, putting into question the importance of word order information.
Schrödinger's Tree -- On Syntax and Neural Language Models
In the last half-decade, the field of natural language processing (NLP) has undergone two major transitions: the switch to neural networks as the primary modeling paradigm and the homogenization of the training regime (pre-train, then fine-tune).
Can Language Models Encode Perceptual Structure Without Grounding? A Case Study in Color
Pretrained language models have been shown to encode relational information, such as the relations between entities or concepts in knowledge-bases -- (Paris, Capital, France).
Attention Can Reflect Syntactic Structure (If You Let It)
Since the popularization of the Transformer as a general-purpose feature encoder for NLP, many studies have attempted to decode linguistic structure from its novel multi-head attention mechanism.
Positional Artefacts Propagate Through Masked Language Model Embeddings
In this work, we demonstrate that the contextualized word vectors derived from pretrained masked language model-based encoders share a common, perhaps undesirable pattern across layers.
K\opsala: Transition-Based Graph Parsing via Efficient Training and Effective Encoding
We present K{\o}psala, the Copenhagen-Uppsala system for the Enhanced Universal Dependencies Shared Task at IWPT 2020.
Køpsala: Transition-Based Graph Parsing via Efficient Training and Effective Encoding
We present K{\o}psala, the Copenhagen-Uppsala system for the Enhanced Universal Dependencies Shared Task at IWPT 2020.
Do Neural Language Models Show Preferences for Syntactic Formalisms?
Recent work on the interpretability of deep neural language models has concluded that many properties of natural language syntax are encoded in their representational spaces.
Deep Contextualized Word Embeddings in Transition-Based and Graph-Based Dependency Parsing - A Tale of Two Parsers Revisited
Transition-based and graph-based dependency parsers have previously been shown to have complementary strengths and weaknesses: transition-based parsers exploit rich structural features but suffer from error propagation, while graph-based parsers benefit from global optimization but have restricted feature scope.
Higher-order Comparisons of Sentence Encoder Representations
Representational Similarity Analysis (RSA) is a technique developed by neuroscientists for comparing activity patterns of different measurement modalities (e. g., fMRI, electrophysiology, behavior).
Deep Contextualized Word Embeddings in Transition-Based and Graph-Based Dependency Parsing -- A Tale of Two Parsers Revisited
Transition-based and graph-based dependency parsers have previously been shown to have complementary strengths and weaknesses: transition-based parsers exploit rich structural features but suffer from error propagation, while graph-based parsers benefit from global optimization but have restricted feature scope.
What can we learn from Semantic Tagging?
We investigate the effects of multi-task learning using the recently introduced task of semantic tagging.
AffecThor at SemEval-2018 Task 1: A cross-linguistic approach to sentiment intensity quantification in tweets
In this paper we describe our submission to SemEval-2018 Task 1: Affects in Tweets.
Discriminator at SemEval-2018 Task 10: Minimally Supervised Discrimination
We participated to the SemEval-2018 shared task on capturing discriminative attributes (Task 10) with a simple system that ranked 8th amongst the 26 teams that took part in the evaluation.
The Power of Character N-grams in Native Language Identification
In this paper, we explore the performance of a linear SVM trained on language independent character features for the NLI Shared Task 2017.
