Search Results for author:
Found 46 papers, 6 papers with code
Knowledge Distillation for Swedish NER models: A Search for Performance and Efficiency
The field encompasses different methods that aim to preserve the performance of a model while decreasing the size of it.
Can We Use Small Models to Investigate Multimodal Fusion Methods?
This is a setting in which we fuse language with information from the math modality and strive to replicate some fusion methods from the vision-and-language domain.
Natural Language Processing in Policy Evaluation: Extracting Policy Conditions from IMF Loan Agreements
Social science researchers often use text as the raw data in investigations: for instance, when investigating the effects of IMF policies on the development of countries under IMF programs, researchers typically encode structured descriptions of the programs using a time-consuming manual effort.
Can Large Language Models (or Humans) Disentangle Text?
We investigate the potential of large language models (LLMs) to disentangle text variables--to remove the textual traces of an undesired forbidden variable in a task sometimes known as text distillation and closely related to the fairness in AI and causal inference literature.
What Happens to a Dataset Transformed by a Projection-based Concept Removal Method?
We investigate the behavior of methods that use linear projections to remove information about a concept from a language representation, and we consider the question of what happens to a dataset transformed by such a method.
The Effect of Scaling, Retrieval Augmentation and Form on the Factual Consistency of Language Models
In this work, we identify potential causes of inconsistency and evaluate the effectiveness of two mitigation strategies: up-scaling and augmenting the LM with a retrieval corpus.
Surface-Based Retrieval Reduces Perplexity of Retrieval-Augmented Language Models
Inspired by this, we replace the semantic retrieval in Retro with a surface-level method based on BM25, obtaining a significant reduction in perplexity.
An Empirical Study of Multitask Learning to Improve Open Domain Dialogue Systems
Autoregressive models used to generate responses in open-domain dialogue systems often struggle to take long-term context into account and to maintain consistency over a dialogue.
On the Generalization Ability of Retrieval-Enhanced Transformers
Recent work on the Retrieval-Enhanced Transformer (RETRO) model has shown that off-loading memory from trainable weights to a retrieval database can significantly improve language modeling and match the performance of non-retrieval models that are an order of magnitude larger in size.
Controlling for Stereotypes in Multimodal Language Model Evaluation
We propose a methodology and design two benchmark sets for measuring to what extent language-and-vision language models use the visual signal in the presence or absence of stereotypes.
How to Adapt Pre-trained Vision-and-Language Models to a Text-only Input?
To find the best approach, we investigate and compare seven possible methods for adapting three different pre-trained VL models to text-only input.
What do Models Learn From Training on More Than Text? Measuring Visual Commonsense Knowledge
Primarily, we find that the visual commonsense knowledge is not significantly different between the multimodal models and unimodal baseline models trained on visual text data.
Conceptualizing Treatment Leakage in Text-based Causal Inference
However, these methods rely on a critical assumption that there is no treatment leakage: that is, the text only contains information about the confounder and no information about treatment assignment.
Transferring Knowledge from Vision to Language: How to Achieve it and how to Measure it?
We find that our method can successfully be used to measure visual knowledge transfer capabilities in models and that our novel model architecture shows promising results for leveraging multimodal knowledge in a unimodal setting.
Training a Swedish Constituency Parser on Six Incompatible Treebanks
We investigate a transition-based parser that uses Eukalyptus, a function-tagged constituent treebank for Swedish which includes discontinuous constituents.
An Arabic Tweets Sentiment Analysis Dataset (ATSAD) using Distant Supervision and Self Training
As the number of social media users increases, they express their thoughts, needs, socialise and publish their opinions reviews.
The 2018 Shared Task on Extrinsic Parser Evaluation: On the Downstream Utility of English Universal Dependency Parsers
We summarize empirical results and tentative conclusions from the Second Extrinsic Parser Evaluation Initiative (EPE 2018).
Automatically Linking Lexical Resources with Word Sense Embedding Models
Automatically learnt word sense embeddings are developed as an attempt to refine the capabilities of coarse word embeddings.
Training Word Sense Embeddings With Lexicon-based Regularization
We propose to improve word sense embeddings by enriching an automatic corpus-based method with lexicographic data.
Character-based recurrent neural networks for morphological relational reasoning
We also show that using the auxiliary task of learning the relation type speeds up convergence and improves the prediction accuracy for the word generation task.
Retrieving Occurrences of Grammatical Constructions
Finding authentic examples of grammatical constructions is central in constructionist approaches to linguistics, language processing, and second language learning.
Romanized Berber and Romanized Arabic Automatic Language Identification Using Machine Learning
The ALI standard methods require datasets for training and use character/word-based n-gram models.
Automatic Detection of Arabicized Berber and Arabic Varieties
Automatic Language Identification (ALI) is the detection of the natural language of an input text by a machine.
ASIREM Participation at the Discriminating Similar Languages Shared Task 2016
In sub-task 2, which deals with Arabic dialect identification, the system achieved its best performance using character-based n-grams (49. 67{\%} accuracy), ranking fourth in the closed track (the best result being 51. 16{\%}), and an accuracy of 53. 18{\%}, ranking first in the open track.
A Multi-domain Corpus of Swedish Word Sense Annotation
We describe the word sense annotation layer in \textit{Eukalyptus}, a freely available five-domain corpus of contemporary Swedish with several annotation layers.
Gulf Arabic Linguistic Resource Building for Sentiment Analysis
To fill this gap, we created these two main linguistic resources.
A Simple and Efficient Method To Generate Word Sense Representations
Distributed representations of words have boosted the performance of many Natural Language Processing tasks.
Semantic Role Labeling with the Swedish FrameNet
We present the first results on semantic role labeling using the Swedish FrameNet, which is a lexical resource currently in development.
Improving the Recall of a Discourse Parser by Constraint-based Postprocessing
We describe two constraint-based methods that can be used to improve the recall of a shallow discourse parser based on conditional random field chunking.
