no code implementations • WS (NoDaLiDa) 2019 • Yuri Bizzoni, Marius Mosbach, Dietrich Klakow, Stefania Degaetano-Ortlieb
We apply hyperbolic embeddings to trace the dynamics of change of conceptual-semantic relationships in a large diachronic scientific corpus (200 years).
no code implementations • LREC 2022 • Tugtekin Turan, Dietrich Klakow, Emmanuel Vincent, Denis Jouvet
In recent years, voice-controlled personal assistants have revolutionized the interaction with smart devices and mobile applications.
no code implementations • EMNLP (BlackboxNLP) 2020 • Tejaswani Verma, Christoph Lingenfelder, Dietrich Klakow
With the increase in the use of AI systems, a need for explanation systems arises.
no code implementations • RANLP 2021 • Marius Mosbach, Irina Stenger, Tania Avgustinova, Bernd Möbius, Dietrich Klakow
We present an extended version of a tool developed for calculating linguistic distances and asymmetries in auditory perception of closely related languages.
no code implementations • VarDial (COLING) 2022 • Iuliia Zaitova, Badr Abdullah, Dietrich Klakow
Closely related languages are often mutually intelligible to various degrees.
no code implementations • LREC 2022 • Irina Stenger, Philip Georgis, Tania Avgustinova, Bernd Möbius, Dietrich Klakow
We focus on the syntactic variation and measure syntactic distances between nine Slavic languages (Belarusian, Bulgarian, Croatian, Czech, Polish, Slovak, Slovene, Russian, and Ukrainian) using symmetric measures of insertion, deletion and movement of syntactic units in the parallel sentences of the fable “The North Wind and the Sun”.
no code implementations • LEGAL (LREC) 2022 • Mickaël Rigault, Claudia Cevenini, Khalid Choukri, Martin Kocour, Karel Veselý, Igor Szoke, Petr Motlicek, Juan Pablo Zuluaga-Gomez, Alexander Blatt, Dietrich Klakow, Allan Tart, Pavel Kolčárek, Jan Černocký
In this paper the authors detail the various legal and ethical issues faced during the ATCO2 project.
no code implementations • ISA (LREC) 2022 • Jutta Stock, Volha Petukhova, Dietrich Klakow
We hypothesise that the ISO 24617-2 dialogue act annotation framework adequately supports sales negotiation assessment in the domain of call centre conversations.
no code implementations • RANLP 2021 • Ekaterina Saveleva, Volha Petukhova, Marius Mosbach, Dietrich Klakow
The paper presents a novel discourse-based approach to argument quality assessment defined as a graph classification task, where the depth of reasoning (argumentation) is evident from the number and type of detected discourse units and relations between them.
no code implementations • EMNLP (insights) 2020 • Ashwin Geet D’Sa, Irina Illina, Dominique Fohr, Dietrich Klakow, Dana Ruiter
In this paper, label propagation-based semi-supervised learning is explored for the task of hate speech classification.
no code implementations • ACL (ISA, IWCS) 2021 • Ekaterina Saveleva, Volha Petukhova, Marius Mosbach, Dietrich Klakow
We tested the widely used Penn Discourse Tree Bank full parser (Lin et al., 2010) and the state-of-the-art neural network NeuralEDUSeg (Wang et al., 2018) and XLNet (Yang et al., 2019) models on the two-stage discourse segmentation and discourse relation recognition.
no code implementations • 22 Apr 2024 • Dawei Zhu, Pinzhen Chen, Miaoran Zhang, Barry Haddow, Xiaoyu Shen, Dietrich Klakow
Traditionally, success in multilingual machine translation can be attributed to three key factors in training data: large volume, diverse translation directions, and high quality.
no code implementations • 17 Apr 2024 • Dawei Zhu, Sony Trenous, Xiaoyu Shen, Dietrich Klakow, Bill Byrne, Eva Hasler
Recent research has shown that large language models (LLMs) can achieve remarkable translation performance through supervised fine-tuning (SFT) using only a small amount of parallel data.
2 code implementations • 4 Apr 2024 • Vagrant Gautam, Eileen Bingert, Dawei Zhu, Anne Lauscher, Dietrich Klakow
Robust, faithful and harm-free pronoun use for individuals is an important goal for language models as their use increases, but prior work tends to study only one or two of these characteristics at a time.
1 code implementation • 1 Apr 2024 • Miaoran Zhang, Mingyang Wang, Jesujoba O. Alabi, Dietrich Klakow
This paper presents our system developed for the SemEval-2024 Task 1: Semantic Textual Relatedness for African and Asian Languages.
no code implementations • 20 Mar 2024 • Atnafu Lambebo Tonja, Israel Abebe Azime, Tadesse Destaw Belay, Mesay Gemeda Yigezu, Moges Ahmed Mehamed, Abinew Ali Ayele, Ebrahim Chekol Jibril, Michael Melese Woldeyohannis, Olga Kolesnikova, Philipp Slusallek, Dietrich Klakow, Shengwu Xiong, Seid Muhie Yimam
We open-source our multilingual language models, new benchmark datasets for various downstream tasks, and task-specific fine-tuned language models and discuss the performance of the models.
no code implementations • 20 Mar 2024 • Paloma García-de-Herreros, Vagrant Gautam, Philipp Slusallek, Dietrich Klakow, Marius Mosbach
ORCA (Shen et al., 2023) is a recent technique for cross-modal fine-tuning, i. e., applying pre-trained transformer models to modalities beyond their training data.
no code implementations • 20 Feb 2024 • Miaoran Zhang, Vagrant Gautam, Mingyang Wang, Jesujoba O. Alabi, Xiaoyu Shen, Dietrich Klakow, Marius Mosbach
Compared to work on monolingual (English) in-context learning, multilingual in-context learning is under-explored, and we lack an in-depth understanding of the role of demonstrations in this context.
no code implementations • 20 Feb 2024 • Jesujoba O. Alabi, Marius Mosbach, Matan Eyal, Dietrich Klakow, Mor Geva
We analyze the operation of transformer language adapters, which are small modules trained on top of a frozen language model to adapt its predictions to new target languages.
no code implementations • 12 Dec 2023 • Mohammed Maqsood Shaik, Dietrich Klakow, Badr M. Abdullah
To address this challenge, we propose self-supervised adaptive pre-training (SAPT) to adapt the pre-trained model to the target domain and languages of the downstream task.
no code implementations • 18 Nov 2023 • Michael A. Hedderich, Jonas Fischer, Dietrich Klakow, Jilles Vreeken
Characterizing these errors in easily interpretable terms gives insight into whether a classifier is prone to making systematic errors, but also gives a way to act and improve the classifier.
no code implementations • 8 Nov 2023 • Julius Steuer, Marius Mosbach, Dietrich Klakow
Research on the cognitive plausibility of language models (LMs) has so far mostly concentrated on modelling psycholinguistic response variables such as reading times, gaze durations and N400/P600 EEG signals, while mostly leaving out the dimension of what Mahowald et al. (2023) described as formal and functional linguistic competence, and developmental plausibility.
1 code implementation • 30 Oct 2023 • Vagrant Gautam, Miaoran Zhang, Dietrich Klakow
If a question cannot be answered with the available information, robust systems for question answering (QA) should know _not_ to answer.
no code implementations • 9 Aug 2023 • Julius Steuer, Badr Abdullah, Johann-Mattis List, Dietrich Klakow
Training data for our PLMs consists of word lists with a maximum of 1000 entries per language.
no code implementations • 12 Jun 2023 • Aravind Krishnan, Jesujoba Alabi, Dietrich Klakow
This study investigates the potential usage of PLMs for language modelling in ASR.
Automatic Speech Recognition Automatic Speech Recognition (ASR) +3
1 code implementation • 4 Jun 2023 • Badr M. Abdullah, Mohammed Maqsood Shaik, Bernd Möbius, Dietrich Klakow
Self-supervised representation learning for speech often involves a quantization step that transforms the acoustic input into discrete units.
1 code implementation • 27 May 2023 • Dawei Zhu, Xiaoyu Shen, Marius Mosbach, Andreas Stephan, Dietrich Klakow
In this paper, we revisit the setup of these approaches and find that the benefits brought by these approaches are significantly overestimated.
1 code implementation • 26 May 2023 • Marius Mosbach, Tiago Pimentel, Shauli Ravfogel, Dietrich Klakow, Yanai Elazar
In this paper, we compare the generalization of few-shot fine-tuning and in-context learning to challenge datasets, while controlling for the models used, the number of examples, and the number of parameters, ranging from 125M to 30B.
1 code implementation • 23 May 2023 • Cheikh M. Bamba Dione, David Adelani, Peter Nabende, Jesujoba Alabi, Thapelo Sindane, Happy Buzaaba, Shamsuddeen Hassan Muhammad, Chris Chinenye Emezue, Perez Ogayo, Anuoluwapo Aremu, Catherine Gitau, Derguene Mbaye, Jonathan Mukiibi, Blessing Sibanda, Bonaventure F. P. Dossou, Andiswa Bukula, Rooweither Mabuya, Allahsera Auguste Tapo, Edwin Munkoh-Buabeng, Victoire Memdjokam Koagne, Fatoumata Ouoba Kabore, Amelia Taylor, Godson Kalipe, Tebogo Macucwa, Vukosi Marivate, Tajuddeen Gwadabe, Mboning Tchiaze Elvis, Ikechukwu Onyenwe, Gratien Atindogbe, Tolulope Adelani, Idris Akinade, Olanrewaju Samuel, Marien Nahimana, Théogène Musabeyezu, Emile Niyomutabazi, Ester Chimhenga, Kudzai Gotosa, Patrick Mizha, Apelete Agbolo, Seydou Traore, Chinedu Uchechukwu, Aliyu Yusuf, Muhammad Abdullahi, Dietrich Klakow
In this paper, we present MasakhaPOS, the largest part-of-speech (POS) dataset for 20 typologically diverse African languages.
no code implementations • 31 Mar 2023 • Idris Akinade, Jesujoba Alabi, David Adelani, Clement Odoje, Dietrich Klakow
This paper investigates the performance of massively multilingual neural machine translation (NMT) systems in translating Yor\`ub\'a greetings ($\varepsilon$ k\'u [MASK]), which are a big part of Yor\`ub\'a language and culture, into English.
Cultural Vocal Bursts Intensity Prediction Machine Translation +2
no code implementations • 8 Jan 2023 • Badr M. Abdullah, Dietrich Klakow
In this paper, we take a closer analytical look at AWEs learned from English speech and study how the choice of the learning objective and the architecture shapes their representational profile.
3 code implementations • 8 Nov 2022 • Juan Zuluaga-Gomez, Karel Veselý, Igor Szöke, Alexander Blatt, Petr Motlicek, Martin Kocour, Mickael Rigault, Khalid Choukri, Amrutha Prasad, Seyyed Saeed Sarfjoo, Iuliia Nigmatulina, Claudia Cevenini, Pavel Kolčárek, Allan Tart, Jan Černocký, Dietrich Klakow
In this paper, we introduce the ATCO2 corpus, a dataset that aims at fostering research on the challenging ATC field, which has lagged behind due to lack of annotated data.
Automatic Speech Recognition Automatic Speech Recognition (ASR) +6
1 code implementation • 22 Oct 2022 • David Ifeoluwa Adelani, Graham Neubig, Sebastian Ruder, Shruti Rijhwani, Michael Beukman, Chester Palen-Michel, Constantine Lignos, Jesujoba O. Alabi, Shamsuddeen H. Muhammad, Peter Nabende, Cheikh M. Bamba Dione, Andiswa Bukula, Rooweither Mabuya, Bonaventure F. P. Dossou, Blessing Sibanda, Happy Buzaaba, Jonathan Mukiibi, Godson Kalipe, Derguene Mbaye, Amelia Taylor, Fatoumata Kabore, Chris Chinenye Emezue, Anuoluwapo Aremu, Perez Ogayo, Catherine Gitau, Edwin Munkoh-Buabeng, Victoire M. Koagne, Allahsera Auguste Tapo, Tebogo Macucwa, Vukosi Marivate, Elvis Mboning, Tajuddeen Gwadabe, Tosin Adewumi, Orevaoghene Ahia, Joyce Nakatumba-Nabende, Neo L. Mokono, Ignatius Ezeani, Chiamaka Chukwuneke, Mofetoluwa Adeyemi, Gilles Q. Hacheme, Idris Abdulmumin, Odunayo Ogundepo, Oreen Yousuf, Tatiana Moteu Ngoli, Dietrich Klakow
African languages are spoken by over a billion people, but are underrepresented in NLP research and development.
no code implementations • 19 Oct 2022 • Anupama Chingacham, Vera Demberg, Dietrich Klakow
In noisy environments, speech can be hard to understand for humans.
1 code implementation • 14 Sep 2022 • Badr M. Abdullah, Bernd Möbius, Dietrich Klakow
Models of acoustic word embeddings (AWEs) learn to map variable-length spoken word segments onto fixed-dimensionality vector representations such that different acoustic exemplars of the same word are projected nearby in the embedding space.
1 code implementation • 4 Aug 2022 • Vilém Zouhar, Marius Mosbach, Dietrich Klakow
We present an LSTM-based autoregressive language model which uses prefix embeddings (from a pretrained masked language model) via fusion (e. g. concatenation) to obtain a richer context representation for language modelling.
1 code implementation • 15 Jun 2022 • Ali Davody, David Ifeoluwa Adelani, Thomas Kleinbauer, Dietrich Klakow
Transferring knowledge from one domain to another is of practical importance for many tasks in natural language processing, especially when the amount of available data in the target domain is limited.
no code implementations • 3 Jun 2022 • Dawei Zhu, Michael A. Hedderich, Fangzhou Zhai, David Ifeoluwa Adelani, Dietrich Klakow
However, text classification in low-resource languages is still challenging due to the lack of annotated data.
1 code implementation • NAACL (WOAH) 2022 • Awantee Deshpande, Dana Ruiter, Marius Mosbach, Dietrich Klakow
Analyzing ethnic or religious bias is important for improving fairness, accountability, and transparency of natural language processing models.
1 code implementation • 20 May 2022 • Lukas Lange, Jannik Strötgen, Heike Adel, Dietrich Klakow
The detection and normalization of temporal expressions is an important task and preprocessing step for many applications.
1 code implementation • NAACL (SocialNLP) 2022 • Dana Ruiter, Thomas Kleinbauer, Cristina España-Bonet, Josef van Genabith, Dietrich Klakow
Recent research on style transfer takes inspiration from unsupervised neural machine translation (UNMT), learning from large amounts of non-parallel data by exploiting cycle consistency loss, back-translation, and denoising autoencoders.
1 code implementation • 15 May 2022 • Dawei Zhu, Xiaoyu Shen, Michael A. Hedderich, Dietrich Klakow
Training deep neural networks (DNNs) under weak supervision has attracted increasing research attention as it can significantly reduce the annotation cost.
1 code implementation • NAACL 2022 • David Ifeoluwa Adelani, Jesujoba Oluwadara Alabi, Angela Fan, Julia Kreutzer, Xiaoyu Shen, Machel Reid, Dana Ruiter, Dietrich Klakow, Peter Nabende, Ernie Chang, Tajuddeen Gwadabe, Freshia Sackey, Bonaventure F. P. Dossou, Chris Chinenye Emezue, Colin Leong, Michael Beukman, Shamsuddeen Hassan Muhammad, Guyo Dub Jarso, Oreen Yousuf, Andre Niyongabo Rubungo, Gilles Hacheme, Eric Peter Wairagala, Muhammad Umair Nasir, Benjamin Ayoade Ajibade, Tunde Oluwaseyi Ajayi, Yvonne Wambui Gitau, Jade Abbott, Mohamed Ahmed, Millicent Ochieng, Anuoluwapo Aremu, Perez Ogayo, Jonathan Mukiibi, Fatoumata Ouoba Kabore, Godson Koffi Kalipe, Derguene Mbaye, Allahsera Auguste Tapo, Victoire Memdjokam Koagne, Edwin Munkoh-Buabeng, Valencia Wagner, Idris Abdulmumin, Ayodele Awokoya, Happy Buzaaba, Blessing Sibanda, Andiswa Bukula, Sam Manthalu
We focus on two questions: 1) How can pre-trained models be used for languages not included in the initial pre-training?
1 code implementation • LREC 2022 • Dana Ruiter, Liane Reiners, Ashwin Geet D'Sa, Thomas Kleinbauer, Dominique Fohr, Irina Illina, Dietrich Klakow, Christian Schemer, Angeliki Monnier
Even though hate speech (HS) online has been an important object of research in the last decade, most HS-related corpora over-simplify the phenomenon of hate by attempting to label user comments as "hate" or "neutral".
1 code implementation • NAACL 2022 • Miaoran Zhang, Marius Mosbach, David Ifeoluwa Adelani, Michael A. Hedderich, Dietrich Klakow
Learning semantically meaningful sentence embeddings is an open problem in natural language processing.
1 code implementation • insights (ACL) 2022 • Dawei Zhu, Michael A. Hedderich, Fangzhou Zhai, David Ifeoluwa Adelani, Dietrich Klakow
Incorrect labels in training data occur when human annotators make mistakes or when the data is generated via weak or distant supervision.
1 code implementation • COLING 2022 • Jesujoba O. Alabi, David Ifeoluwa Adelani, Marius Mosbach, Dietrich Klakow
Multilingual pre-trained language models (PLMs) have demonstrated impressive performance on several downstream tasks for both high-resourced and low-resourced languages.
no code implementations • 13 Apr 2022 • Alexander Blatt, Martin Kocour, Karel Veselý, Igor Szöke, Dietrich Klakow
The introduced data augmentation adds additional performance on high WER transcripts and allows the adaptation of the model to unseen airspaces.
1 code implementation • SpaNLP (ACL) 2022 • Vilém Zouhar, Marius Mosbach, Miaoran Zhang, Dietrich Klakow
Finally, we show that it is possible to combine PCA with using 1bit per dimension.
no code implementations • AKBC Workshop CSKB 2021 • Vilém Zouhar, Marius Mosbach, Debanjali Biswas, Dietrich Klakow
Many NLP models gain performance by having access to a knowledge base.
1 code implementation • 16 Dec 2021 • Lukas Lange, Heike Adel, Jannik Strötgen, Dietrich Klakow
The field of natural language processing (NLP) has recently seen a large change towards using pre-trained language models for solving almost any task.
no code implementations • 27 Oct 2021 • Florian Dietz, Dietrich Klakow
Even though most interfaces in the real world are discrete, no efficient way exists to train neural networks to make use of them, yet.
2 code implementations • 18 Oct 2021 • Michael Hedderich, Jonas Fischer, Dietrich Klakow, Jilles Vreeken
Characterizing these errors in easily interpretable terms gives insight into whether a classifier is prone to making systematic errors, but also gives a way to act and improve the classifier.
1 code implementation • EMNLP (BlackboxNLP) 2021 • Badr M. Abdullah, Iuliia Zaitova, Tania Avgustinova, Bernd Möbius, Dietrich Klakow
We further discuss the implications of our work on modeling speech processing and language similarity with neural networks.
1 code implementation • EMNLP 2021 • David Ifeoluwa Adelani, Miaoran Zhang, Xiaoyu Shen, Ali Davody, Thomas Kleinbauer, Dietrich Klakow
Documents as short as a single sentence may inadvertently reveal sensitive information about their authors, including e. g. their gender or ethnicity.
no code implementations • MTSummit 2021 • Dana Ruiter, Dietrich Klakow, Josef van Genabith, Cristina España-Bonet
For most language combinations, parallel data is either scarce or simply unavailable.
no code implementations • 18 Jul 2021 • Anupama Chingacham, Vera Demberg, Dietrich Klakow
We evaluate the intelligibility of synonyms in context and find that choosing a lexical unit that is less risky to be misheard than its synonym introduced an average gain in comprehension of 37% at SNR -5 dB and 21% at SNR 0 dB for babble noise.
no code implementations • 8 Jul 2021 • Michael A. Hedderich, Benjamin Roth, Katharina Kann, Barbara Plank, Alex Ratner, Dietrich Klakow
Welcome to WeaSuL 2021, the First Workshop on Weakly Supervised Learning, co-located with ICLR 2021.
1 code implementation • 16 Jun 2021 • Badr M. Abdullah, Marius Mosbach, Iuliia Zaitova, Bernd Möbius, Dietrich Klakow
Our experiments show that (1) the distance in the embedding space in the best cases only moderately correlates with phonological distance, and (2) improving the performance on the word discrimination task does not necessarily yield models that better reflect word phonological similarity.
1 code implementation • ACL (WOAH) 2021 • Vanessa Hahn, Dana Ruiter, Thomas Kleinbauer, Dietrich Klakow
We observe that, on both similar and distant target tasks and across all languages, the subspace-based representations transfer more effectively than standard BERT representations in the zero-shot setting, with improvements between F1 +10. 9 and F1 +42. 9 over the baselines across all tested monolingual and cross-lingual scenarios.
1 code implementation • EMNLP 2021 • Lukas Lange, Jannik Strötgen, Heike Adel, Dietrich Klakow
For this, we study the effects of model transfer on sequence labeling across various domains and tasks and show that our methods based on model similarity and support vector machines are able to predict promising sources, resulting in performance increases of up to 24 F1 points.
no code implementations • EACL 2021 • Alexandra Mayn, Badr M. Abdullah, Dietrich Klakow
We present a deep neural model of spoken word recognition which is trained to retrieve the meaning of a word (in the form of a word embedding) given its spoken form, a task which resembles that faced by a human listener.
no code implementations • EACL 2021 • Nicole Macher, Badr M. Abdullah, Harm Brouwer, Dietrich Klakow
Theories and models of spoken word recognition aim to explain the process of accessing lexical knowledge given an acoustic realization of a word form.
1 code implementation • 25 Feb 2021 • Michael A. Hedderich, Lukas Lange, Dietrich Klakow
Distant supervision allows obtaining labeled training corpora for low-resource settings where only limited hand-annotated data exists.
Low Resource Named Entity Recognition named-entity-recognition +2
1 code implementation • EACL 2021 • Susann Boy, Dana Ruiter, Dietrich Klakow
This is done using a transfer learning approach, where the parameters learned by an emoji-based source task are transferred to a sentiment target task.
3 code implementations • 24 Jan 2021 • Michael A. Hedderich, Dawei Zhu, Dietrich Klakow
Distant and weak supervision allow to obtain large amounts of labeled training data quickly and cheaply, but these automatic annotations tend to contain a high amount of errors.
no code implementations • SEMEVAL 2020 • Kathryn Chapman, Johannes Bernhard, Dietrich Klakow
We present our submission and results for SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020) where we participated in offensive tweet classification tasks in English, Arabic, Greek, Turkish and Danish.
1 code implementation • COLING 2020 • Marius Mosbach, Stefania Degaetano-Ortlieb, Marie-Pauline Krielke, Badr M. Abdullah, Dietrich Klakow
Transformer-based language models achieve high performance on various tasks, but we still lack understanding of the kind of linguistic knowledge they learn and rely on.
no code implementations • 28 Oct 2020 • Marimuthu Kalimuthu, Aditya Mogadala, Marius Mosbach, Dietrich Klakow
Building on these recent developments, and with the aim of improving the quality of generated captions, the contribution of our work in this paper is two-fold: First, we propose a generic multimodal model fusion framework for caption generation as well as emendation where we utilize different fusion strategies to integrate a pretrained Auxiliary Language Model (AuxLM) within the traditional encoder-decoder visual captioning frameworks.
Automatic Speech Recognition Automatic Speech Recognition (ASR) +5
1 code implementation • NAACL 2021 • Michael A. Hedderich, Lukas Lange, Heike Adel, Jannik Strötgen, Dietrich Klakow
Deep neural networks and huge language models are becoming omnipresent in natural language applications.
1 code implementation • EMNLP 2021 • Lukas Lange, Heike Adel, Jannik Strötgen, Dietrich Klakow
Combining several embeddings typically improves performance in downstream tasks as different embeddings encode different information.
no code implementations • VarDial (COLING) 2020 • Badr M. Abdullah, Jacek Kudera, Tania Avgustinova, Bernd Möbius, Dietrich Klakow
In this paper, we present a neural model for Slavic language identification in speech signals and analyze its emergent representations to investigate whether they reflect objective measures of language relatedness and/or non-linguists' perception of language similarity.
1 code implementation • EMNLP 2020 • Michael A. Hedderich, David Adelani, Dawei Zhu, Jesujoba Alabi, Udia Markus, Dietrich Klakow
Multilingual transformer models like mBERT and XLM-RoBERTa have obtained great improvements for many NLP tasks on a variety of languages.
no code implementations • EMNLP (BlackboxNLP) 2020 • Marius Mosbach, Anna Khokhlova, Michael A. Hedderich, Dietrich Klakow
Our analysis reveals that while fine-tuning indeed changes the representations of a pre-trained model and these changes are typically larger for higher layers, only in very few cases, fine-tuning has a positive effect on probing accuracy that is larger than just using the pre-trained model with a strong pooling method.
1 code implementation • EMNLP 2020 • Moritz Wolf, Dana Ruiter, Ashwin Geet D'Sa, Liane Reiners, Jan Alexandersson, Dietrich Klakow
A lot of real-world phenomena are complex and cannot be captured by single task annotations.
1 code implementation • 7 Aug 2020 • David Ifeoluwa Adelani, Ali Davody, Thomas Kleinbauer, Dietrich Klakow
Machine Learning approaches to Natural Language Processing tasks benefit from a comprehensive collection of real-life user data.
1 code implementation • 2 Aug 2020 • Badr M. Abdullah, Tania Avgustinova, Bernd Möbius, Dietrich Klakow
State-of-the-art spoken language identification (LID) systems, which are based on end-to-end deep neural networks, have shown remarkable success not only in discriminating between distant languages but also between closely-related languages or even different spoken varieties of the same language.
no code implementations • 22 Jul 2020 • Aditya Mogadala, Xiaoyu Shen, Dietrich Klakow
Particularly, these image features are subdivided into global and local features, where global features are extracted from the global representation of the image, while local features are extracted from the objects detected locally in an image.
no code implementations • 12 Jul 2020 • Aditya Mogadala, Marius Mosbach, Dietrich Klakow
Generating longer textual sequences when conditioned on the visual information is an interesting problem to explore.
1 code implementation • 19 Jun 2020 • Ali Davody, David Ifeoluwa Adelani, Thomas Kleinbauer, Dietrich Klakow
Differentially private stochastic gradient descent (DPSGD) is a variation of stochastic gradient descent based on the Differential Privacy (DP) paradigm, which can mitigate privacy threats that arise from the presence of sensitive information in training data.
no code implementations • 9 Jun 2020 • Gabriele Bettgenhäuser, Michael A. Hedderich, Dietrich Klakow
Although multitask learning has achieved improved performance in some problems, there are also tasks that lose performance when trained together.
2 code implementations • ICLR 2021 • Marius Mosbach, Maksym Andriushchenko, Dietrich Klakow
Fine-tuning pre-trained transformer-based language models such as BERT has become a common practice dominating leaderboards across various NLP benchmarks.
no code implementations • ACL 2020 • Xiaoyu Shen, Ernie Chang, Hui Su, Jie zhou, Dietrich Klakow
The neural attention model has achieved great success in data-to-text generation tasks.
no code implementations • LREC 2020 • Marc Schulder, Johannah O{'}Mahony, Yury Bakanouski, Dietrich Klakow
In air traffic control, assistant systems support air traffic controllers in their work.
Automatic Speech Recognition Automatic Speech Recognition (ASR) +1
no code implementations • 18 Mar 2020 • David Ifeoluwa Adelani, Michael A. Hedderich, Dawei Zhu, Esther van den Berg, Dietrich Klakow
Techniques such as distant and weak supervision can be used to create labeled data in a (semi-) automatic way.
Low Resource Named Entity Recognition named-entity-recognition +3
no code implementations • 16 Dec 2019 • Dawei Zhu, Aditya Mogadala, Dietrich Klakow
We propose the Two-sidEd Attentive conditional Generative Adversarial Network (TEA-cGAN) to generate semantically manipulated images while preserving other contents such as background intact.
no code implementations • IJCNLP 2019 • Xiaoyu Shen, Yang Zhao, Hui Su, Dietrich Klakow
Pointer Generators have been the de facto standard for modern summarization systems.
1 code implementation • IJCNLP 2019 • Lukas Lange, Michael A. Hedderich, Dietrich Klakow
In low-resource settings, the performance of supervised labeling models can be improved with automatically annotated or distantly supervised data, which is cheap to create but often noisy.
Low Resource Named Entity Recognition named-entity-recognition +4
1 code implementation • IJCNLP 2019 • Xiaoyu Shen, Jun Suzuki, Kentaro Inui, Hui Su, Dietrich Klakow, Satoshi Sekine
As a result, the content to be described in the text cannot be explicitly controlled.
no code implementations • RANLP 2019 • Marius Mosbach, Irina Stenger, Tania Avgustinova, Dietrich Klakow
Languages may be differently distant from each other and their mutual intelligibility may be asymmetric.
no code implementations • RANLP 2019 • Martin Wolf, Volha Petukhova, Dietrich Klakow
The processing of medical information is not a trivial task for medical non-experts.
no code implementations • 22 Jul 2019 • Aditya Mogadala, Marimuthu Kalimuthu, Dietrich Klakow
Interest in Artificial Intelligence (AI) and its applications has seen unprecedented growth in the last few years.
no code implementations • NAACL 2019 • Andrew Johnson, Penny Karanasou, Judith Gaspers, Dietrich Klakow
This work explores cross-lingual transfer learning (TL) for named entity recognition, focusing on bootstrapping Japanese from English.
1 code implementation • WS 2019 • Michael A. Hedderich, Andrew Yates, Dietrich Klakow, Gerard de Melo
However, they typically cannot serve as a drop-in replacement for conventional single-sense embeddings, because the correct sense vector needs to be selected for each word.
no code implementations • NAACL 2019 • Debjit Paul, Mittul Singh, Michael A. Hedderich, Dietrich Klakow
In our experiments on Chunking and NER, this approach performs more robustly than the baselines.
no code implementations • 8 Feb 2019 • Kathrin Grosse, Thomas A. Trost, Marius Mosbach, Michael Backes, Dietrich Klakow
Recently, a weight-based attack on stochastic gradient descent inducing overfitting has been proposed.
no code implementations • WS 2018 • David M. Howcroft, Dietrich Klakow, Vera Demberg
Developing conventional natural language generation systems requires extensive attention from human experts in order to craft complex sets of sentence planning rules.
no code implementations • WS 2018 • Natalia Skachkova, Thomas Trost, Dietrich Klakow
The combination of opening and closing brackets is a typical example of such a construction.
1 code implementation • 29 Oct 2018 • Marius Mosbach, Maksym Andriushchenko, Thomas Trost, Matthias Hein, Dietrich Klakow
Recently, Kannan et al. [2018] proposed several logit regularization methods to improve the adversarial robustness of classifiers.
no code implementations • EMNLP 2018 • Hui Su, Xiaoyu Shen, Wenjie Li, Dietrich Klakow
Sequence-to-Sequence (seq2seq) models have become overwhelmingly popular in building end-to-end trainable dialogue systems.
1 code implementation • WS 2018 • Michael A. Hedderich, Dietrich Klakow
Manually labeled corpora are expensive to create and often not available for low-resource languages or domains.
no code implementations • LREC 2018 • Volha Petukhova, Andrei Malchanau, Youssef Oualil, Dietrich Klakow, Saturnino Luz, Fasih Haider, Nick Campbell, Dimitris Koryzis, Dimitris Spiliotopoulos, Pierre Albert, Nicklas Linz, Alex, Jan ersson
no code implementations • 23 Aug 2017 • Youssef Oualil, Dietrich Klakow
The performance of Neural Network (NN)-based language models is steadily improving due to the emergence of new architectures, which are able to learn different natural language characteristics.
no code implementations • EMNLP 2016 • Youssef Oualil, Mittul Singh, Clayton Greenberg, Dietrich Klakow
The goal of language modeling techniques is to capture the statistical and structural properties of natural languages from training corpora.
1 code implementation • 20 Aug 2017 • Youssef Oualil, Dietrich Klakow
Training large vocabulary Neural Network Language Models (NNLMs) is a difficult task due to the explicit requirement of the output layer normalization, which typically involves the evaluation of the full softmax function over the complete vocabulary.
no code implementations • WS 2017 • Thomas Alex Trost, er, Dietrich Klakow
Word embeddings are high-dimensional vector representations of words and are thus difficult to interpret.
no code implementations • 23 Mar 2017 • Youssef Oualil, Clayton Greenberg, Mittul Singh, Dietrich Klakow
Feedforward Neural Network (FNN)-based language models estimate the probability of the next word based on the history of the last N words, whereas Recurrent Neural Networks (RNN) perform the same task based only on the last word and some context information that cycles in the network.
no code implementations • COLING 2016 • Mittul Singh, Clayton Greenberg, Youssef Oualil, Dietrich Klakow
We augmented pre-trained word embeddings with these novel embeddings and evaluated on a rare word similarity task, obtaining up to 3 times improvement in correlation over the original set of embeddings.
no code implementations • LREC 2016 • Andrea Fischer, Kl{\'a}ra J{\'a}grov{\'a}, Irina Stenger, Tania Avgustinova, Dietrich Klakow, Rol Marti,
In an intercomprehension scenario, typically a native speaker of language L1 is confronted with output from an unknown, but related language L2.
no code implementations • LREC 2016 • Dilafruz Amanova, Volha Petukhova, Dietrich Klakow
This paper describes a method to automatically create dialogue resources annotated with dialogue act information by reusing existing dialogue corpora.
no code implementations • LREC 2014 • Volha Petukhova, Martin Gropp, Dietrich Klakow, Gregor Eigner, Mario Topf, Stefan Srb, Petr Motlicek, Blaise Potard, John Dines, Olivier Deroo, Ronny Egeler, Uwe Meinz, Steffen Liersch, Anna Schmidt
We first start with human-human Wizard of Oz experiments to collect human-human data in order to model natural human dialogue behaviour, for better understanding of phenomena of human interactions and predicting interlocutors actions, and then replace the human Wizard by an increasingly advanced dialogue system, using evaluation data for system improvement.
no code implementations • 6 Jan 2014 • Benjamin Roth, Tassilo Barth, Michael Wiegand, Mittul Singh, Dietrich Klakow
In the TAC KBP 2013 English Slotfilling evaluation, the submitted main run of the LSV RelationFactory system achieved the top-ranked F1-score of 37. 3%.
no code implementations • LREC 2012 • Stasinos Konstantopoulos, Valia Kordoni, Nicola Cancedda, Vangelis Karkaletsis, Dietrich Klakow, Jean-Michel Renders
In this paper we explore a task-driven approach to interfacing NLP components, where language processing is guided by the end-task that each application requires.
no code implementations • LREC 2012 • Michael Wiegand, Benjamin Roth, Eva Lasarcyk, Stephanie Köser, Dietrich Klakow
We present a gold standard for semantic relation extraction in the food domain for German.