Search Results for author:
Found 78 papers, 50 papers with code
End-to-end style-conditioned poetry generation: What does it take to learn from examples alone?
In this work, we design an end-to-end model for poetry generation based on conditioned recurrent neural network (RNN) language models whose goal is to learn stylistic features (poem length, sentiment, alliteration, and rhyming) from examples alone.
Evaluation of Coreference Resolution Systems Under Adversarial Attacks
A substantial overlap of coreferent mentions in the CoNLL dataset magnifies the recent progress on coreference resolution.
TUDA-Reproducibility @ ReproGen: Replicability of Human Evaluation of Text-to-Text and Concept-to-Text Generation
This paper describes our contribution to the Shared Task ReproGen by Belz et al. (2021), which investigates the reproducibility of human evaluations in the context of Natural Language Generation.
TUDa at WMT21: Sentence-Level Direct Assessment with Adapters
This paper presents our submissions to the WMT2021 Shared Task on Quality Estimation, Task 1 Sentence-Level Direct Assessment.
Syntactic Language Change in English and German: Metrics, Parsers, and Convergences
Even though we have evidence that recent parsers trained on modern treebanks are not heavily affected by data 'noise' such as spelling changes and OCR errors in our historic data, we find that results of syntactic language change are sensitive to the parsers involved, which is a caution against using a single parser for evaluating syntactic language change as done in previous work.
Is there really a Citation Age Bias in NLP?
Recently, it has been noted that there is a citation age bias in the Natural Language Processing (NLP) community, one of the currently fastest growing AI subfields, in that the mean age of the bibliography of NLP papers has become ever younger in the last few years, leading to `citation amnesia' in which older knowledge is increasingly forgotten.
NLLG Quarterly arXiv Report 09/23: What are the most influential current AI Papers?
Artificial Intelligence (AI) has witnessed rapid growth, especially in the subfields Natural Language Processing (NLP), Machine Learning (ML) and Computer Vision (CV).
The Eval4NLP 2023 Shared Task on Prompting Large Language Models as Explainable Metrics
Specifically, we propose a novel competition setting in which we select a list of allowed LLMs and disallow fine-tuning to ensure a focus on prompting.
AutomaTikZ: Text-Guided Synthesis of Scientific Vector Graphics with TikZ
To address this, we propose the use of TikZ, a well-known abstract graphics language that can be compiled to vector graphics, as an intermediate representation of scientific figures.
NLLG Quarterly arXiv Report 06/23: What are the most influential current AI Papers?
In particular, we compile a list of the 40 most popular papers based on normalized citation counts from the first half of 2023.
Cross-lingual Cross-temporal Summarization: Dataset, Models, Evaluation
Additionally, we explore the potential of ChatGPT for CLCTS as a summarizer and an evaluator.
Towards Explainable Evaluation Metrics for Machine Translation
In this context, we also discuss the latest state-of-the-art approaches to explainable metrics based on generative models such as ChatGPT and GPT4.
Cross-Genre Argument Mining: Can Language Models Automatically Fill in Missing Discourse Markers?
We demonstrate the impact of our approach on an Argument Mining downstream task, evaluated on different corpora, showing that language models can be trained to automatically fill in discourse markers across different corpora, improving the performance of a downstream model in some, but not all, cases.
Trade-Offs Between Fairness and Privacy in Language Modeling
Protecting privacy in contemporary NLP models is gaining in importance.
Missing Information, Unresponsive Authors, Experimental Flaws: The Impossibility of Assessing the Reproducibility of Previous Human Evaluations in NLP
We report our efforts in identifying a set of previous human evaluations in NLP that would be suitable for a coordinated study examining what makes human evaluations in NLP more/less reproducible.
ChatGPT: A Meta-Analysis after 2.5 Months
ChatGPT, a chatbot developed by OpenAI, has gained widespread popularity and media attention since its release in November 2022.
Transformers Go for the LOLs: Generating (Humourous) Titles from Scientific Abstracts End-to-End
Our human evaluation suggests that our best end-to-end system performs similarly to human authors (but arguably slightly worse).
BMX: Boosting Natural Language Generation Metrics with Explainability
We then combine this segment-level score with the original metric to obtain a better metric.
ByGPT5: End-to-End Style-conditioned Poetry Generation with Token-free Language Models
State-of-the-art poetry generation systems are often complex.
FairGer: Using NLP to Measure Support for Women and Migrants in 155 Years of German Parliamentary Debates
We measure support with women and migrants in German political debates over the last 155 years.
EffEval: A Comprehensive Evaluation of Efficiency for MT Evaluation Metrics
In this work, we provide a comprehensive evaluation of efficiency for MT evaluation metrics.
Layer or Representation Space: What makes BERT-based Evaluation Metrics Robust?
The evaluation of recent embedding-based evaluation metrics for text generation is primarily based on measuring their correlation with human evaluations on standard benchmarks.
MENLI: Robust Evaluation Metrics from Natural Language Inference
Recently proposed BERT-based evaluation metrics for text generation perform well on standard benchmarks but are vulnerable to adversarial attacks, e. g., relating to information correctness.
Reproducibility Issues for BERT-based Evaluation Metrics
We find that reproduction of claims and results often fails because of (i) heavy undocumented preprocessing involved in the metrics, (ii) missing code and (iii) reporting weaker results for the baseline metrics.
Towards Explainable Evaluation Metrics for Natural Language Generation
We also provide a synthesizing overview over recent approaches for explainable machine translation metrics and discuss how they relate to those goals and properties.
Did AI get more negative recently?
We annotate over 1. 5 k papers from NLP and ML to train a SciBERT-based model to automatically predict the stance of a paper based on its title and abstract.
USCORE: An Effective Approach to Fully Unsupervised Evaluation Metrics for Machine Translation
We show that our fully unsupervised metrics are effective, i. e., they beat supervised competitors on 4 out of our 5 evaluation datasets.
Constrained Density Matching and Modeling for Cross-lingual Alignment of Contextualized Representations
Multilingual representations pre-trained with monolingual data exhibit considerably unequal task performances across languages.
DiscoScore: Evaluating Text Generation with BERT and Discourse Coherence
Still, recent BERT-based evaluation metrics are weak in recognizing coherence, and thus are not reliable in a way to spot the discourse-level improvements of those text generation systems.
Better than Average: Paired Evaluation of NLP Systems
Evaluation in NLP is usually done by comparing the scores of competing systems independently averaged over a common set of test instances.
Global Explainability of BERT-Based Evaluation Metrics by Disentangling along Linguistic Factors
Evaluation metrics are a key ingredient for progress of text generation systems.
The Eval4NLP Shared Task on Explainable Quality Estimation: Overview and Results
In this paper, we introduce the Eval4NLP-2021shared task on explainable quality estimation.
Constrained Density Matching and Modeling for Effective Contextualized Alignment
In this work, we analyze the limitations according to which previous alignments become very resource-intensive, \emph{viz.,} (i) the inability to sufficiently leverage data and (ii) that alignments are not trained properly.
Diachronic Analysis of German Parliamentary Proceedings: Ideological Shifts through the Lens of Political Biases
We analyze bias in historical corpora as encoded in diachronic distributional semantic models by focusing on two specific forms of bias, namely a political (i. e., anti-communism) and racist (i. e., antisemitism) one.
Changes in European Solidarity Before and During COVID-19: Evidence from a Large Crowd- and Expert-Annotated Twitter Dataset
We use these annotations to train a BERT model with multiple data augmentation strategies.
Graph Routing between Capsules
Therefore, in this paper, we introduce a new capsule network with graph routing to learn both relationships, where capsules in each layer are treated as the nodes of a graph.
BERT-Defense: A Probabilistic Model Based on BERT to Combat Cognitively Inspired Orthographic Adversarial Attacks
Adversarial attacks expose important blind spots of deep learning systems.
From Hero to Z\'eroe: A Benchmark of Low-Level Adversarial Attacks
Adversarial attacks are label-preserving modifications to inputs of machine learning classifiers designed to fool machines but not humans.
CMCE at SemEval-2020 Task 1: Clustering on Manifolds of Contextualized Embeddings to Detect Historical Meaning Shifts
Remarkably, with only 10 dimensional MBERT embeddings (reduced from the original size of 768), our submitted model performs best on subtask 1 for English and ranks third in subtask 2 for English.
Probing Multilingual BERT for Genetic and Typological Signals
We probe the layers in multilingual BERT (mBERT) for phylogenetic and geographic language signals across 100 languages and compute language distances based on the mBERT representations.
Vec2Sent: Probing Sentence Embeddings with Natural Language Generation
We introspect black-box sentence embeddings by conditionally generating from them with the objective to retrieve the underlying discrete sentence.
From Hero to Zéroe: A Benchmark of Low-Level Adversarial Attacks
Adversarial attacks are label-preserving modifications to inputs of machine learning classifiers designed to fool machines but not humans.
How to Probe Sentence Embeddings in Low-Resource Languages: On Structural Design Choices for Probing Task Evaluation
We then probe embeddings in a multilingual setup with design choices that lie in a 'stable region', as we identify for English, and find that results on English do not transfer to other languages.
SUPERT: Towards New Frontiers in Unsupervised Evaluation Metrics for Multi-Document Summarization
Compared to the state-of-the-art unsupervised evaluation metrics, SUPERT correlates better with human ratings by 18-39%.
On the Limitations of Cross-lingual Encoders as Exposed by Reference-Free Machine Translation Evaluation
We systematically investigate a range of metrics based on state-of-the-art cross-lingual semantic representations obtained with pretrained M-BERT and LASER.
PO-EMO: Conceptualization, Annotation, and Modeling of Aesthetic Emotions in German and English Poetry
Thus, we conceptualize a set of aesthetic emotions that are predictive of aesthetic appreciation in the reader, and allow the annotation of multiple labels per line to capture mixed emotions within their context.
Semantic Change and Emerging Tropes In a Large Corpus of New High German Poetry
Due to its semantic succinctness and novelty of expression, poetry is a great test bed for semantic change analysis.
MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distance
A robust evaluation metric has a profound impact on the development of text generation systems.
Towards Scalable and Reliable Capsule Networks for Challenging NLP Applications
Obstacles hindering the development of capsule networks for challenging NLP applications include poor scalability to large output spaces and less reliable routing processes.
Ranked #1 on
Text Classification
on RCV1
(P@1 metric)
Pitfalls in the Evaluation of Sentence Embeddings
Our motivation is to challenge the current evaluation of sentence embeddings and to provide an easy-to-access reference for future research.
Does My Rebuttal Matter? Insights from a Major NLP Conference
We then focus on the role of the rebuttal phase, and propose a novel task to predict after-rebuttal (i. e., final) scores from initial reviews and author responses.
Text Processing Like Humans Do: Visually Attacking and Shielding NLP Systems
Visual modifications to text are often used to obfuscate offensive comments in social media (e. g., "! d10t") or as a writing style ("1337" in "leet speak"), among other scenarios.
Predicting Research Trends From Arxiv
By extrapolation, we predict that these topics will remain lead problems/approaches in their fields in the short- and mid-term.
Is it Time to Swish? Comparing Deep Learning Activation Functions Across NLP tasks
Activation functions play a crucial role in neural networks because they are the nonlinearities which have been attributed to the success story of deep learning.
PD3: Better Low-Resource Cross-Lingual Transfer By Combining Direct Transfer and Annotation Projection
We consider unsupervised cross-lingual transfer on two tasks, viz., sentence-level argumentation mining and standard POS tagging.
Killing Four Birds with Two Stones: Multi-Task Learning for Non-Literal Language Detection
In this paper we investigate multi-task learning for related non-literal language phenomena.
One Size Fits All? A simple LSTM for non-literal token and construction-level classification
In this paper, we tackle four different tasks of non-literal language classification: token and construction level metaphor detection, classification of idiomatic use of infinitive-verb compounds, and classification of non-literal particle verbs.
Cross-lingual Argumentation Mining: Machine Translation (and a bit of Projection) is All You Need!
Argumentation mining (AM) requires the identification of complex discourse structures and has lately been applied with success monolingually.
ArgumenText: Searching for Arguments in Heterogeneous Sources
Argument mining is a core technology for enabling argument search in large corpora.
Multi-Task Learning for Argumentation Mining in Low-Resource Settings
We investigate whether and where multi-task learning (MTL) can improve performance on NLP problems related to argumentation mining (AM), in particular argument component identification.
Concatenated Power Mean Word Embeddings as Universal Cross-Lingual Sentence Representations
Here, we generalize the concept of average word embeddings to power mean word embeddings.
What is the Essence of a Claim? Cross-Domain Claim Identification
Argument mining has become a popular research area in NLP.
Neural End-to-End Learning for Computational Argumentation Mining
Contrary to models that operate on the argument component level, we find that framing AM as dependency parsing leads to subpar performance results.
On the Linearity of Semantic Change: Investigating Meaning Variation via Dynamic Graph Models
We consider two graph models of semantic change.
EELECTION at SemEval-2017 Task 10: Ensemble of nEural Learners for kEyphrase ClassificaTION
From these approaches, we created an ensemble of differently hyper-parameterized systems, achieving a micro-F1-score of 0. 63 on the test data.
Still not there? Comparing Traditional Sequence-to-Sequence Models to Encoder-Decoder Neural Networks on Monotone String Translation Tasks
We analyze the performance of encoder-decoder neural models and compare them with well-known established methods.
Language classification from bilingual word embedding graphs
We study the role of the second language in bilingual word embeddings in monolingual semantic evaluation tasks.
Lemmatization and Morphological Tagging in German and Latin: A Comparison and a Survey of the State-of-the-art
This paper relates to the challenge of morphological tagging and lemmatization in morphologically rich languages by example of German and Latin.
Complex Decomposition of the Negative Distance kernel
Our evaluation shows that the power kernel produces F-scores that are comparable to the reference kernels, but is -- except for the linear kernel -- faster to compute.
On the Number of Many-to-Many Alignments of Multiple Sequences
We provide a new asymptotic formula for the case $S=\{(s_1,\ldots, s_N) \:|\: 1\le s_i\le 2\}$.
