Search Results for author:
Found 27 papers, 12 papers with code
Some steps towards the generation of diachronic WordNets
We apply hyperbolic embeddings to trace the dynamics of change of conceptual-semantic relationships in a large diachronic scientific corpus (200 years).
Discourse-based Argument Segmentation and Annotation
We tested the widely used Penn Discourse Tree Bank full parser (Lin et al., 2010) and the state-of-the-art neural network NeuralEDUSeg (Wang et al., 2018) and XLNet (Yang et al., 2019) models on the two-stage discourse segmentation and discourse relation recognition.
incom.py 2.0 - Calculating Linguistic Distances and Asymmetries in Auditory Perception of Closely Related Languages
We present an extended version of a tool developed for calculating linguistic distances and asymmetries in auditory perception of closely related languages.
Graph-based Argument Quality Assessment
The paper presents a novel discourse-based approach to argument quality assessment defined as a graph classification task, where the depth of reasoning (argumentation) is evident from the number and type of detected discourse units and relations between them.
LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders
We outperform encoder-only models by a large margin on word-level tasks and reach a new unsupervised state-of-the-art performance on the Massive Text Embeddings Benchmark (MTEB).
What explains the success of cross-modal fine-tuning with ORCA?
ORCA (Shen et al., 2023) is a recent technique for cross-modal fine-tuning, i. e., applying pre-trained transformer models to modalities beyond their training data.
The Impact of Demonstrations on Multilingual In-Context Learning: A Multidimensional Analysis
Compared to work on monolingual (English) in-context learning, multilingual in-context learning is under-explored, and we lack an in-depth understanding of the role of demonstrations in this context.
The Hidden Space of Transformer Language Adapters
We analyze the operation of transformer language adapters, which are small modules trained on top of a frozen language model to adapt its predictions to new target languages.
Large GPT-like Models are Bad Babies: A Closer Look at the Relationship between Linguistic Competence and Psycholinguistic Measures
Research on the cognitive plausibility of language models (LMs) has so far mostly concentrated on modelling psycholinguistic response variables such as reading times, gaze durations and N400/P600 EEG signals, while mostly leaving out the dimension of what Mahowald et al. (2023) described as formal and functional linguistic competence, and developmental plausibility.
Weaker Than You Think: A Critical Look at Weakly Supervised Learning
In this paper, we revisit the setup of these approaches and find that the benefits brought by these approaches are significantly overestimated.
Few-shot Fine-tuning vs. In-context Learning: A Fair Comparison and Evaluation
In this paper, we compare the generalization of few-shot fine-tuning and in-context learning to challenge datasets, while controlling for the models used, the number of examples, and the number of parameters, ranging from 125M to 30B.
Fusing Sentence Embeddings Into LSTM-based Autoregressive Language Models
We present an LSTM-based autoregressive language model which uses prefix embeddings (from a pretrained masked language model) via fusion (e. g. concatenation) to obtain a richer context representation for language modelling.
Measuring Causal Effects of Data Statistics on Language Model's `Factual' Predictions
Our causal framework and our results demonstrate the importance of studying datasets and the benefits of causality for understanding NLP models.
StereoKG: Data-Driven Knowledge Graph Construction for Cultural Knowledge and Stereotypes
Analyzing ethnic or religious bias is important for improving fairness, accountability, and transparency of natural language processing models.
MCSE: Multimodal Contrastive Learning of Sentence Embeddings
Learning semantically meaningful sentence embeddings is an open problem in natural language processing.
Adapting Pre-trained Language Models to African Languages via Multilingual Adaptive Fine-Tuning
Multilingual pre-trained language models (PLMs) have demonstrated impressive performance on several downstream tasks for both high-resourced and low-resourced languages.
Knowledge Base Index Compression via Dimensionality and Precision Reduction
Finally, we show that it is possible to combine PCA with using 1bit per dimension.
Artefact Retrieval: Overview of NLP Models with Knowledge Base Access
Many NLP models gain performance by having access to a knowledge base.
Do Acoustic Word Embeddings Capture Phonological Similarity? An Empirical Study
Our experiments show that (1) the distance in the embedding space in the best cases only moderately correlates with phonological distance, and (2) improving the performance on the word discrimination task does not necessarily yield models that better reflect word phonological similarity.
A Closer Look at Linguistic Knowledge in Masked Language Models: The Case of Relative Clauses in American English
Transformer-based language models achieve high performance on various tasks, but we still lack understanding of the kind of linguistic knowledge they learn and rely on.
Fusion Models for Improved Visual Captioning
Building on these recent developments, and with the aim of improving the quality of generated captions, the contribution of our work in this paper is two-fold: First, we propose a generic multimodal model fusion framework for caption generation as well as emendation where we utilize different fusion strategies to integrate a pretrained Auxiliary Language Model (AuxLM) within the traditional encoder-decoder visual captioning frameworks.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+5
On the Interplay Between Fine-tuning and Sentence-level Probing for Linguistic Knowledge in Pre-trained Transformers
Our analysis reveals that while fine-tuning indeed changes the representations of a pre-trained model and these changes are typically larger for higher layers, only in very few cases, fine-tuning has a positive effect on probing accuracy that is larger than just using the pre-trained model with a strong pooling method.
Sparse Graph to Sequence Learning for Vision Conditioned Long Textual Sequence Generation
Generating longer textual sequences when conditioned on the visual information is an interesting problem to explore.
On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines
Fine-tuning pre-trained transformer-based language models such as BERT has become a common practice dominating leaderboards across various NLP benchmarks.
incom.py - A Toolbox for Calculating Linguistic Distances and Asymmetries between Related Languages
Languages may be differently distant from each other and their mutual intelligibility may be asymmetric.
On the security relevance of weights in deep learning
Recently, a weight-based attack on stochastic gradient descent inducing overfitting has been proposed.
Logit Pairing Methods Can Fool Gradient-Based Attacks
Recently, Kannan et al. [2018] proposed several logit regularization methods to improve the adversarial robustness of classifiers.
