Search Results for author: Hinrich Schütze

Found 197 papers, 93 papers with code

Don’t Forget Cheap Training Signals Before Building Unsupervised Bilingual Word Embeddings

no code implementations LREC (BUCC) 2022 Silvia Severini, Viktor Hangya, Masoud Jalili Sabet, Alexander Fraser, Hinrich Schütze

The two approaches we find most effective are: 1) using identical words as seed lexicons (which unsupervised approaches incorrectly assume are not available for orthographically distinct language pairs) and 2) combining such lexicons with pairs extracted by matching romanized versions of words with an edit distance threshold.

Cross-Lingual Transfer Word Embeddings

Separating Hate Speech and Offensive Language Classes via Adversarial Debiasing

1 code implementation NAACL (WOAH) 2022 Shuzhou Yuan, Antonis Maronikolakis, Hinrich Schütze

Research to tackle hate speech plaguing online media has made strides in providing solutions, analyzing bias and curating data.

Wine is not v i n. On the Compatibility of Tokenizations across Languages

no code implementations Findings (EMNLP) 2021 Antonis Maronikolakis, Philipp Dufter, Hinrich Schütze

The size of the vocabulary is a central design choice in large pretrained language models, with respect to both performance and memory requirements.

Multidomain Pretrained Language Models for Green NLP

1 code implementation EACL (AdaptNLP) 2021 Antonis Maronikolakis, Hinrich Schütze

Thus, instead of training multiple models, we can train a single multidomain model saving on computational resources and training time.

Domain Adaptation

Few-Shot Text Generation with Natural Language Instructions

no code implementations EMNLP 2021 Timo Schick, Hinrich Schütze

Providing pretrained language models with simple task descriptions in natural language enables them to solve some tasks in a fully unsupervised fashion.

Headline Generation text-classification +1

MaiBaam: A Multi-Dialectal Bavarian Universal Dependency Treebank

no code implementations15 Mar 2024 Verena Blaschke, Barbara Kovačić, Siyao Peng, Hinrich Schütze, Barbara Plank

Despite the success of the Universal Dependencies (UD) project exemplified by its impressive language breadth, there is still a lack in `within-language breadth': most treebanks focus on standard languages.

POS POS Tagging

Hybrid Human-LLM Corpus Construction and LLM Evaluation for Rare Linguistic Phenomena

no code implementations11 Mar 2024 Leonie Weissweiler, Abdullatif Köksal, Hinrich Schütze

Argument Structure Constructions (ASCs) are one of the most well-studied construction groups, providing a unique opportunity to demonstrate the usefulness of Construction Grammar (CxG).

Dependency Parsing Sentence

Decomposed Prompting: Unveiling Multilingual Linguistic Structure Knowledge in English-Centric Large Language Models

no code implementations28 Feb 2024 Ercong Nie, Shuzhou Yuan, Bolei Ma, Helmut Schmid, Michael Färber, Frauke Kreuter, Hinrich Schütze

Despite the predominance of English in their training data, English-centric Large Language Models (LLMs) like GPT-3 and LLaMA display a remarkable ability to perform multilingual tasks, raising questions about the depth and nature of their cross-lingual capabilities.

Part-Of-Speech Tagging Sentence

Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Models

1 code implementation26 Feb 2024 Paul Röttger, Valentin Hofmann, Valentina Pyatkin, Musashi Hinck, Hannah Rose Kirk, Hinrich Schütze, Dirk Hovy

Motivated by this discrepancy, we challenge the prevailing constrained evaluation paradigm for values and opinions in LLMs and explore more realistic unconstrained evaluations.

Multiple-choice

GNNavi: Navigating the Information Flow in Large Language Models by Graph Neural Network

no code implementations18 Feb 2024 Shuzhou Yuan, Ercong Nie, Michael Färber, Helmut Schmid, Hinrich Schütze

Large Language Models (LLMs) exhibit strong In-Context Learning (ICL) capabilities when prompts with demonstrations are applied to them.

In-Context Learning text-classification +1

ToPro: Token-Level Prompt Decomposition for Cross-Lingual Sequence Labeling Tasks

1 code implementation29 Jan 2024 Bolei Ma, Ercong Nie, Shuzhou Yuan, Helmut Schmid, Michael Färber, Frauke Kreuter, Hinrich Schütze

However, most previous studies primarily focused on sentence-level classification tasks, and only a few considered token-level labeling tasks such as Named Entity Recognition (NER) and Part-of-Speech (POS) tagging.

Benchmarking In-Context Learning +8

HiFT: A Hierarchical Full Parameter Fine-Tuning Strategy

no code implementations26 Jan 2024 Yongkang Liu, Yiqun Zhang, Qian Li, Tong Liu, Shi Feng, Daling Wang, Yifei Zhang, Hinrich Schütze

As LMs grow in size, fine-tuning the full parameters of LMs requires a prohibitively large amount of GPU memory.

TransliCo: A Contrastive Learning Framework to Address the Script Barrier in Multilingual Pretrained Language Models

1 code implementation12 Jan 2024 Yihong Liu, Chunlan Ma, Haotian Ye, Hinrich Schütze

As a result, mPLMs present a script barrier: representations from different scripts are located in different subspaces, which is a strong indicator of why crosslingual transfer involving languages of different scripts shows sub-optimal performance.

Contrastive Learning Transliteration

MoSECroT: Model Stitching with Static Word Embeddings for Crosslingual Zero-shot Transfer

no code implementations9 Jan 2024 Haotian Ye, Yihong Liu, Chunlan Ma, Hinrich Schütze

In this paper, we introduce MoSECroT Model Stitching with Static Word Embeddings for Crosslingual Zero-shot Transfer), a novel and challenging task that is especially relevant to low-resource languages for which static word embeddings are available.

Word Embeddings

Multilingual Word Embeddings for Low-Resource Languages using Anchors and a Chain of Related Languages

no code implementations21 Nov 2023 Viktor Hangya, Silvia Severini, Radoslav Ralev, Alexander Fraser, Hinrich Schütze

In this paper, we propose to build multilingual word embeddings (MWEs) via a novel language chain-based approach, that incorporates intermediate related languages to bridge the gap between the distant source and target.

Bilingual Lexicon Induction Multilingual NLP +1

OFA: A Framework of Initializing Unseen Subword Embeddings for Efficient Large-scale Multilingual Continued Pretraining

no code implementations15 Nov 2023 Yihong Liu, Peiqin Lin, Mingyang Wang, Hinrich Schütze

Therefore, a more efficient method is to adapt existing pretrained language models (PLMs) to new languages via vocabulary extension and continued pretraining.

Language Modelling Multilingual Word Embeddings

GlotLID: Language Identification for Low-Resource Languages

3 code implementations24 Oct 2023 Amir Hossein Kargaran, Ayyoob Imani, François Yvon, Hinrich Schütze

Several recent papers have published good solutions for language identification (LID) for about 300 high-resource and medium-resource languages.

Dialect Identification

GradSim: Gradient-Based Language Grouping for Effective Multilingual Training

1 code implementation23 Oct 2023 Mingyang Wang, Heike Adel, Lukas Lange, Jannik Strötgen, Hinrich Schütze

However, not all languages positively influence each other and it is an open research question how to select the most suitable set of languages for multilingual training and avoid negative interference among languages whose characteristics or data distributions are not compatible.

Sentiment Analysis

Unleashing the Multilingual Encoder Potential: Boosting Zero-Shot Performance via Probability Calibration

1 code implementation8 Oct 2023 Ercong Nie, Helmut Schmid, Hinrich Schütze

Pretrained multilingual encoder models can directly perform zero-shot multilingual tasks or linguistic probing by reformulating the input examples into cloze-style prompts.

Position

GlotScript: A Resource and Tool for Low Resource Writing System Identification

1 code implementation23 Sep 2023 Amir Hossein Kargaran, François Yvon, Hinrich Schütze

We present GlotScript, an open resource and tool for low resource writing system identification.

Language Modelling

Cross-Lingual Constituency Parsing for Middle High German: A Delexicalized Approach

no code implementations9 Aug 2023 Ercong Nie, Helmut Schmid, Hinrich Schütze

However, training an automatic syntactic analysis system for ancient languages solely relying on annotated parse data is a formidable task due to the inherent challenges in building treebanks for such languages.

Constituency Parsing Cross-Lingual Transfer

Is Prompt-Based Finetuning Always Better than Vanilla Finetuning? Insights from Cross-Lingual Language Understanding

1 code implementation15 Jul 2023 Bolei Ma, Ercong Nie, Helmut Schmid, Hinrich Schütze

We conduct comprehensive experiments on diverse cross-lingual language understanding tasks (sentiment classification, paraphrase identification, and natural language inference) and empirically analyze the variation trends of prompt-based finetuning performance in cross-lingual transfer across different few-shot and full-data settings.

Natural Language Inference Natural Language Understanding +4

On the Copying Problem of Unsupervised NMT: A Training Schedule with a Language Discriminator Loss

1 code implementation26 May 2023 Yihong Liu, Alexandra Chronopoulou, Hinrich Schütze, Alexander Fraser

By conducting extensive experiments on different language pairs, including similar and distant, high and low-resource languages, we find that our method alleviates the copying problem, thus improving the translation performance on low-resource languages.

Machine Translation NMT +2

Evaluate What You Can't Evaluate: Unassessable Quality for Generated Response

no code implementations24 May 2023 Yongkang Liu, Shi Feng, Daling Wang, Yifei Zhang, Hinrich Schütze

There are risks in using eference-free evaluators based on LLMs to evaluate the quality of dialogue responses.

Dialogue Generation

RET-LLM: Towards a General Read-Write Memory for Large Language Models

1 code implementation23 May 2023 Ali Modarressi, Ayyoob Imani, Mohsen Fayyaz, Hinrich Schütze

Large language models (LLMs) have significantly advanced the field of natural language processing (NLP) through their extensive parameters and comprehensive data utilization.

Question Answering

mPLM-Sim: Better Cross-Lingual Similarity and Transfer in Multilingual Pretrained Language Models

1 code implementation23 May 2023 Peiqin Lin, Chengzhi Hu, Zheyu Zhang, André F. T. Martins, Hinrich Schütze

Recent multilingual pretrained language models (mPLMs) have been shown to encode strong language-specific signals, which are not explicitly provided during pretraining.

Open-Ended Question Answering Zero-Shot Cross-Lingual Transfer

Language-Agnostic Bias Detection in Language Models with Bias Probing

no code implementations22 May 2023 Abdullatif Köksal, Omer Faruk Yalcin, Ahmet Akbiyik, M. Tahir Kilavuz, Anna Korhonen, Hinrich Schütze

For nationality as a case study, we show that LABDet `surfaces' nationality bias by training a classifier on top of a frozen PLM on non-nationality sentiment detection.

Bias Detection

A study of conceptual language similarity: comparison and evaluation

no code implementations22 May 2023 Haotian Ye, Yihong Liu, Hinrich Schütze

An interesting line of research in natural language processing (NLP) aims to incorporate linguistic typology to bridge linguistic diversity and assist the research of low-resource languages.

Binary Classification

A Crosslingual Investigation of Conceptualization in 1335 Languages

3 code implementations15 May 2023 Yihong Liu, Haotian Ye, Leonie Weissweiler, Philipp Wicke, Renhao Pei, Robert Zangenfeind, Hinrich Schütze

The resulting measure for the conceptual similarity of two languages is complementary to standard genealogical, typological, and surface similarity measures.

Taxi1500: A Multilingual Dataset for Text Classification in 1500 Languages

no code implementations15 May 2023 Chunlan Ma, Ayyoob ImaniGooghari, Haotian Ye, Ehsaneddin Asgari, Hinrich Schütze

While natural language processing tools have been developed extensively for some of the world's languages, a significant portion of the world's over 7000 languages are still neglected.

text-classification Text Classification

NLNDE at SemEval-2023 Task 12: Adaptive Pretraining and Source Language Selection for Low-Resource Multilingual Sentiment Analysis

no code implementations28 Apr 2023 Mingyang Wang, Heike Adel, Lukas Lange, Jannik Strötgen, Hinrich Schütze

In this work, we propose to leverage language-adaptive and task-adaptive pretraining on African texts and study transfer learning with source language selection on top of an African language-centric pretrained language model.

Language Modelling Sentiment Analysis +1

Does Manipulating Tokenization Aid Cross-Lingual Transfer? A Study on POS Tagging for Non-Standardized Languages

5 code implementations20 Apr 2023 Verena Blaschke, Hinrich Schütze, Barbara Plank

This can for instance be observed when finetuning PLMs on one language and evaluating them on data in a closely related language variety with no standardized orthography.

Cross-Lingual Transfer Part-Of-Speech Tagging +2

A Survey of Corpora for Germanic Low-Resource Languages and Dialects

2 code implementations19 Apr 2023 Verena Blaschke, Hinrich Schütze, Barbara Plank

In this work, we instead focus on low-resource languages and in particular non-standardized low-resource languages.

Sociocultural knowledge is needed for selection of shots in hate speech detection tasks

no code implementations4 Apr 2023 Antonis Maronikolakis, Abdullatif Köksal, Hinrich Schütze

We introduce HATELEXICON, a lexicon of slurs and targets of hate speech for the countries of Brazil, Germany, India and Kenya, to aid training and interpretability of models.

Few-Shot Learning Hate Speech Detection

Construction Grammar Provides Unique Insight into Neural Language Models

no code implementations4 Feb 2023 Leonie Weissweiler, Taiqi He, Naoki Otani, David R. Mortensen, Lori Levin, Hinrich Schütze

Construction Grammar (CxG) has recently been used as the basis for probing studies that have investigated the performance of large pretrained language models (PLMs) with respect to the structure and meaning of constructions.

Position

Cross-Lingual Retrieval Augmented Prompt for Low-Resource Languages

1 code implementation19 Dec 2022 Ercong Nie, Sheng Liang, Helmut Schmid, Hinrich Schütze

Multilingual Pretrained Language Models (MPLMs) have shown their strong multilinguality in recent empirical cross-lingual transfer studies.

Cross-Lingual Transfer Natural Language Inference +3

MEAL: Stable and Active Learning for Few-Shot Prompting

1 code implementation15 Nov 2022 Abdullatif Köksal, Timo Schick, Hinrich Schütze

Few-shot classification has made great strides due to foundation models that, through priming and prompting, are highly effective few-shot learners.

Active Learning Few-Shot Learning +1

Graph-Based Multilingual Label Propagation for Low-Resource Part-of-Speech Tagging

1 code implementation18 Oct 2022 Ayyoob Imani, Silvia Severini, Masoud Jalili Sabet, François Yvon, Hinrich Schütze

An established method for training a POS tagger in such a scenario is to create a labeled training set by transferring from high-resource languages.

Part-Of-Speech Tagging POS +3

Federated Continual Learning for Text Classification via Selective Inter-client Transfer

1 code implementation12 Oct 2022 Yatin Chaudhary, Pranav Rai, Matthias Schubert, Hinrich Schütze, Pankaj Gupta

The objective of Federated Continual Learning (FCL) is to improve deep learning models over life time at each client by (relevant and efficient) knowledge transfer without sharing data.

Continual Learning Federated Learning +3

Modeling Content-Emotion Duality via Disentanglement for Empathetic Conversation

1 code implementation26 Sep 2022 Peiqin Lin, Jiashuo Wang, Hinrich Schütze, Wenjie Li

To solve the task, it is essential to model the content-emotion duality of a dialogue, which is composed of the content view (i. e., what personal experiences are described) and the emotion view (i. e., the feelings of the speaker on these experiences).

Disentanglement Empathetic Response Generation +1

Measuring Causal Effects of Data Statistics on Language Model's `Factual' Predictions

no code implementations28 Jul 2022 Yanai Elazar, Nora Kassner, Shauli Ravfogel, Amir Feder, Abhilasha Ravichander, Marius Mosbach, Yonatan Belinkov, Hinrich Schütze, Yoav Goldberg

Our causal framework and our results demonstrate the importance of studying datasets and the benefits of causality for understanding NLP models.

Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models

3 code implementations9 Jun 2022 Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R. Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, Agnieszka Kluska, Aitor Lewkowycz, Akshat Agarwal, Alethea Power, Alex Ray, Alex Warstadt, Alexander W. Kocurek, Ali Safaya, Ali Tazarv, Alice Xiang, Alicia Parrish, Allen Nie, Aman Hussain, Amanda Askell, Amanda Dsouza, Ambrose Slone, Ameet Rahane, Anantharaman S. Iyer, Anders Andreassen, Andrea Madotto, Andrea Santilli, Andreas Stuhlmüller, Andrew Dai, Andrew La, Andrew Lampinen, Andy Zou, Angela Jiang, Angelica Chen, Anh Vuong, Animesh Gupta, Anna Gottardi, Antonio Norelli, Anu Venkatesh, Arash Gholamidavoodi, Arfa Tabassum, Arul Menezes, Arun Kirubarajan, Asher Mullokandov, Ashish Sabharwal, Austin Herrick, Avia Efrat, Aykut Erdem, Ayla Karakaş, B. Ryan Roberts, Bao Sheng Loe, Barret Zoph, Bartłomiej Bojanowski, Batuhan Özyurt, Behnam Hedayatnia, Behnam Neyshabur, Benjamin Inden, Benno Stein, Berk Ekmekci, Bill Yuchen Lin, Blake Howald, Bryan Orinion, Cameron Diao, Cameron Dour, Catherine Stinson, Cedrick Argueta, César Ferri Ramírez, Chandan Singh, Charles Rathkopf, Chenlin Meng, Chitta Baral, Chiyu Wu, Chris Callison-Burch, Chris Waites, Christian Voigt, Christopher D. Manning, Christopher Potts, Cindy Ramirez, Clara E. Rivera, Clemencia Siro, Colin Raffel, Courtney Ashcraft, Cristina Garbacea, Damien Sileo, Dan Garrette, Dan Hendrycks, Dan Kilman, Dan Roth, Daniel Freeman, Daniel Khashabi, Daniel Levy, Daniel Moseguí González, Danielle Perszyk, Danny Hernandez, Danqi Chen, Daphne Ippolito, Dar Gilboa, David Dohan, David Drakard, David Jurgens, Debajyoti Datta, Deep Ganguli, Denis Emelin, Denis Kleyko, Deniz Yuret, Derek Chen, Derek Tam, Dieuwke Hupkes, Diganta Misra, Dilyar Buzan, Dimitri Coelho Mollo, Diyi Yang, Dong-Ho Lee, Dylan Schrader, Ekaterina Shutova, Ekin Dogus Cubuk, Elad Segal, Eleanor Hagerman, Elizabeth Barnes, Elizabeth Donoway, Ellie Pavlick, Emanuele Rodola, Emma Lam, Eric Chu, Eric Tang, Erkut Erdem, Ernie Chang, Ethan A. Chi, Ethan Dyer, Ethan Jerzak, Ethan Kim, Eunice Engefu Manyasi, Evgenii Zheltonozhskii, Fanyue Xia, Fatemeh Siar, Fernando Martínez-Plumed, Francesca Happé, Francois Chollet, Frieda Rong, Gaurav Mishra, Genta Indra Winata, Gerard de Melo, Germán Kruszewski, Giambattista Parascandolo, Giorgio Mariani, Gloria Wang, Gonzalo Jaimovitch-López, Gregor Betz, Guy Gur-Ari, Hana Galijasevic, Hannah Kim, Hannah Rashkin, Hannaneh Hajishirzi, Harsh Mehta, Hayden Bogar, Henry Shevlin, Hinrich Schütze, Hiromu Yakura, Hongming Zhang, Hugh Mee Wong, Ian Ng, Isaac Noble, Jaap Jumelet, Jack Geissinger, Jackson Kernion, Jacob Hilton, Jaehoon Lee, Jaime Fernández Fisac, James B. Simon, James Koppel, James Zheng, James Zou, Jan Kocoń, Jana Thompson, Janelle Wingfield, Jared Kaplan, Jarema Radom, Jascha Sohl-Dickstein, Jason Phang, Jason Wei, Jason Yosinski, Jekaterina Novikova, Jelle Bosscher, Jennifer Marsh, Jeremy Kim, Jeroen Taal, Jesse Engel, Jesujoba Alabi, Jiacheng Xu, Jiaming Song, Jillian Tang, Joan Waweru, John Burden, John Miller, John U. Balis, Jonathan Batchelder, Jonathan Berant, Jörg Frohberg, Jos Rozen, Jose Hernandez-Orallo, Joseph Boudeman, Joseph Guerr, Joseph Jones, Joshua B. Tenenbaum, Joshua S. Rule, Joyce Chua, Kamil Kanclerz, Karen Livescu, Karl Krauth, Karthik Gopalakrishnan, Katerina Ignatyeva, Katja Markert, Kaustubh D. Dhole, Kevin Gimpel, Kevin Omondi, Kory Mathewson, Kristen Chiafullo, Ksenia Shkaruta, Kumar Shridhar, Kyle McDonell, Kyle Richardson, Laria Reynolds, Leo Gao, Li Zhang, Liam Dugan, Lianhui Qin, Lidia Contreras-Ochando, Louis-Philippe Morency, Luca Moschella, Lucas Lam, Lucy Noble, Ludwig Schmidt, Luheng He, Luis Oliveros Colón, Luke Metz, Lütfi Kerem Şenel, Maarten Bosma, Maarten Sap, Maartje ter Hoeve, Maheen Farooqi, Manaal Faruqui, Mantas Mazeika, Marco Baturan, Marco Marelli, Marco Maru, Maria Jose Ramírez Quintana, Marie Tolkiehn, Mario Giulianelli, Martha Lewis, Martin Potthast, Matthew L. Leavitt, Matthias Hagen, Mátyás Schubert, Medina Orduna Baitemirova, Melody Arnaud, Melvin McElrath, Michael A. Yee, Michael Cohen, Michael Gu, Michael Ivanitskiy, Michael Starritt, Michael Strube, Michał Swędrowski, Michele Bevilacqua, Michihiro Yasunaga, Mihir Kale, Mike Cain, Mimee Xu, Mirac Suzgun, Mitch Walker, Mo Tiwari, Mohit Bansal, Moin Aminnaseri, Mor Geva, Mozhdeh Gheini, Mukund Varma T, Nanyun Peng, Nathan A. Chi, Nayeon Lee, Neta Gur-Ari Krakover, Nicholas Cameron, Nicholas Roberts, Nick Doiron, Nicole Martinez, Nikita Nangia, Niklas Deckers, Niklas Muennighoff, Nitish Shirish Keskar, Niveditha S. Iyer, Noah Constant, Noah Fiedel, Nuan Wen, Oliver Zhang, Omar Agha, Omar Elbaghdadi, Omer Levy, Owain Evans, Pablo Antonio Moreno Casares, Parth Doshi, Pascale Fung, Paul Pu Liang, Paul Vicol, Pegah Alipoormolabashi, Peiyuan Liao, Percy Liang, Peter Chang, Peter Eckersley, Phu Mon Htut, Pinyu Hwang, Piotr Miłkowski, Piyush Patil, Pouya Pezeshkpour, Priti Oli, Qiaozhu Mei, Qing Lyu, Qinlang Chen, Rabin Banjade, Rachel Etta Rudolph, Raefer Gabriel, Rahel Habacker, Ramon Risco, Raphaël Millière, Rhythm Garg, Richard Barnes, Rif A. Saurous, Riku Arakawa, Robbe Raymaekers, Robert Frank, Rohan Sikand, Roman Novak, Roman Sitelew, Ronan LeBras, Rosanne Liu, Rowan Jacobs, Rui Zhang, Ruslan Salakhutdinov, Ryan Chi, Ryan Lee, Ryan Stovall, Ryan Teehan, Rylan Yang, Sahib Singh, Saif M. Mohammad, Sajant Anand, Sam Dillavou, Sam Shleifer, Sam Wiseman, Samuel Gruetter, Samuel R. Bowman, Samuel S. Schoenholz, Sanghyun Han, Sanjeev Kwatra, Sarah A. Rous, Sarik Ghazarian, Sayan Ghosh, Sean Casey, Sebastian Bischoff, Sebastian Gehrmann, Sebastian Schuster, Sepideh Sadeghi, Shadi Hamdan, Sharon Zhou, Shashank Srivastava, Sherry Shi, Shikhar Singh, Shima Asaadi, Shixiang Shane Gu, Shubh Pachchigar, Shubham Toshniwal, Shyam Upadhyay, Shyamolima, Debnath, Siamak Shakeri, Simon Thormeyer, Simone Melzi, Siva Reddy, Sneha Priscilla Makini, Soo-Hwan Lee, Spencer Torene, Sriharsha Hatwar, Stanislas Dehaene, Stefan Divic, Stefano Ermon, Stella Biderman, Stephanie Lin, Stephen Prasad, Steven T. Piantadosi, Stuart M. Shieber, Summer Misherghi, Svetlana Kiritchenko, Swaroop Mishra, Tal Linzen, Tal Schuster, Tao Li, Tao Yu, Tariq Ali, Tatsu Hashimoto, Te-Lin Wu, Théo Desbordes, Theodore Rothschild, Thomas Phan, Tianle Wang, Tiberius Nkinyili, Timo Schick, Timofei Kornev, Titus Tunduny, Tobias Gerstenberg, Trenton Chang, Trishala Neeraj, Tushar Khot, Tyler Shultz, Uri Shaham, Vedant Misra, Vera Demberg, Victoria Nyamai, Vikas Raunak, Vinay Ramasesh, Vinay Uday Prabhu, Vishakh Padmakumar, Vivek Srikumar, William Fedus, William Saunders, William Zhang, Wout Vossen, Xiang Ren, Xiaoyu Tong, Xinran Zhao, Xinyi Wu, Xudong Shen, Yadollah Yaghoobzadeh, Yair Lakretz, Yangqiu Song, Yasaman Bahri, Yejin Choi, Yichi Yang, Yiding Hao, Yifu Chen, Yonatan Belinkov, Yu Hou, Yufang Hou, Yuntao Bai, Zachary Seid, Zhuoye Zhao, Zijian Wang, Zijie J. Wang, ZiRui Wang, Ziyi Wu

BIG-bench focuses on tasks that are believed to be beyond the capabilities of current language models.

Common Sense Reasoning Math +1

Don't Forget Cheap Training Signals Before Building Unsupervised Bilingual Word Embeddings

no code implementations31 May 2022 Silvia Severini, Viktor Hangya, Masoud Jalili Sabet, Alexander Fraser, Hinrich Schütze

The two approaches we find most effective are: 1) using identical words as seed lexicons (which unsupervised approaches incorrectly assume are not available for orthographically distinct language pairs) and 2) combining such lexicons with pairs extracted by matching romanized versions of words with an edit distance threshold.

Cross-Lingual Transfer Word Embeddings

Analyzing Hate Speech Data along Racial, Gender and Intersectional Axes

no code implementations NAACL (GeBNLP) 2022 Antonis Maronikolakis, Philip Baader, Hinrich Schütze

To tackle the rising phenomenon of hate speech, efforts have been made towards data curation and analysis.

Flow-Adapter Architecture for Unsupervised Machine Translation

no code implementations ACL 2022 Yihong Liu, Haris Jabbar, Hinrich Schütze

The primary novelties of our model are: (a) capturing language-specific sentence representations separately for each language using normalizing flows and (b) using a simple transformation of these latent representations for translating from one language to another.

NMT Sentence +2

CaMEL: Case Marker Extraction without Labels

1 code implementation ACL 2022 Leonie Weissweiler, Valentin Hofmann, Masoud Jalili Sabet, Hinrich Schütze

We introduce CaMEL (Case Marker Extraction without Labels), a novel and challenging task in computational morphology that is especially relevant for low-resource languages.

ECOLA: Enhanced Temporal Knowledge Embeddings with Contextualized Language Representations

no code implementations17 Mar 2022 Zhen Han, Ruotong Liao, Jindong Gu, Yao Zhang, Zifeng Ding, Yujia Gu, Heinz Köppl, Hinrich Schütze, Volker Tresp

Since conventional knowledge embedding models cannot take full advantage of the abundant textual information, there have been extensive research efforts in enhancing knowledge embedding using texts.

Knowledge Graph Embedding Link Prediction +1

Geographic Adaptation of Pretrained Language Models

no code implementations16 Mar 2022 Valentin Hofmann, Goran Glavaš, Nikola Ljubešić, Janet B. Pierrehumbert, Hinrich Schütze

While pretrained language models (PLMs) have been shown to possess a plethora of linguistic knowledge, the existing body of research has largely neglected extralinguistic knowledge, which is generally difficult to obtain by pretraining on text alone.

Language Identification Language Modelling +2

Semantic-Oriented Unlabeled Priming for Large-Scale Language Models

no code implementations12 Feb 2022 Yanchen Liu, Timo Schick, Hinrich Schütze

Due to the high costs associated with finetuning large language models, various recent works propose to adapt them to specific tasks without any parameter updates through in-context learning.

In-Context Learning

Towards a Broad Coverage Named Entity Resource: A Data-Efficient Approach for Many Diverse Languages

no code implementations LREC 2022 Silvia Severini, Ayyoob Imani, Philipp Dufter, Hinrich Schütze

Prior work on extracting MNE datasets from parallel corpora required resources such as large monolingual corpora or word aligners that are unavailable or perform poorly for underresourced languages.

Bilingual Lexicon Induction Transliteration

BeliefBank: Adding Memory to a Pre-Trained Language Model for a Systematic Notion of Belief

no code implementations EMNLP 2021 Nora Kassner, Oyvind Tafjord, Hinrich Schütze, Peter Clark

We show that, in a controlled experimental setting, these two mechanisms result in more consistent beliefs in the overall system, improving both the accuracy and consistency of its answers over time.

Language Modelling World Knowledge

Active Learning for Argument Mining: A Practical Approach

no code implementations28 Sep 2021 Nikolai Solmsdorf, Dietrich Trautmann, Hinrich Schütze

Despite considerable recent progress, the creation of well-balanced and diverse resources remains a time-consuming and costly challenge in Argument Mining.

Active Learning Argument Mining

Scene Graph Generation for Better Image Captioning?

no code implementations23 Sep 2021 Maximilian Mozes, Martin Schmitt, Vladimir Golkov, Hinrich Schütze, Daniel Cremers

We investigate the incorporation of visual relationships into the task of supervised image caption generation by proposing a model that leverages detected objects and auto-generated visual relationships to describe images in natural language.

Graph Generation Image Captioning +1

BERT Cannot Align Characters

no code implementations EMNLP (insights) 2021 Antonis Maronikolakis, Philipp Dufter, Hinrich Schütze

We show that the closer two languages are, the better BERT can align them on the character level.

Locating Language-Specific Information in Contextualized Embeddings

1 code implementation16 Sep 2021 Sheng Liang, Philipp Dufter, Hinrich Schütze

Multilingual pretrained language models (MPLMs) exhibit multilinguality and are well suited for transfer across languages.

Graph Algorithms for Multiparallel Word Alignment

1 code implementation EMNLP 2021 Ayyoob Imani, Masoud Jalili Sabet, Lütfi Kerem Şenel, Philipp Dufter, François Yvon, Hinrich Schütze

With the advent of end-to-end deep learning approaches in machine translation, interest in word alignments initially decreased; however, they have again become a focus of research more recently.

Link Prediction Machine Translation +3

Wine is Not v i n. -- On the Compatibility of Tokenizations Across Languages

no code implementations13 Sep 2021 Antonis Maronikolakis, Philipp Dufter, Hinrich Schütze

The size of the vocabulary is a central design choice in large pretrained language models, with respect to both performance and memory requirements.

Continuous Entailment Patterns for Lexical Inference in Context

1 code implementation EMNLP 2021 Martin Schmitt, Hinrich Schütze

If we allow for tokens outside the PLM's vocabulary, patterns can be adapted more flexibly to a PLM's idiosyncrasies.

Few-Shot NLI Lexical Entailment +1

Discrete and Soft Prompting for Multilingual Models

1 code implementation EMNLP 2021 Mengjie Zhao, Hinrich Schütze

It has been shown for English that discrete and soft prompting perform strongly in few-shot learning with pretrained language models (PLMs).

Few-Shot Learning Natural Language Inference

ParCourE: A Parallel Corpus Explorer for a Massively Multilingual Corpus

no code implementations ACL 2021 Ayyoob Imani, Masoud Jalili Sabet, Philipp Dufter, Michael Cysouw, Hinrich Schütze

With more than 7000 languages worldwide, multilingual natural language processing (NLP) is essential both from an academic and commercial perspective.

Multilingual NLP Transfer Learning

Modeling Ideological Salience and Framing in Polarized Online Groups with Graph Neural Networks and Structured Sparsity

1 code implementation Findings (NAACL) 2022 Valentin Hofmann, Xiaowen Dong, Janet B. Pierrehumbert, Hinrich Schütze

The increasing polarization of online political discourse calls for computational tools that automatically detect and monitor ideological divides in social media.

Multi-source Neural Topic Modeling in Multi-view Embedding Spaces

1 code implementation NAACL 2021 Pankaj Gupta, Yatin Chaudhary, Hinrich Schütze

Though word embeddings and topics are complementary representations, several past works have only used pretrained word embeddings in (neural) topic modeling to address data sparsity in short-text or small collection of documents.

Information Retrieval Retrieval +1

Generating Datasets with Pretrained Language Models

2 code implementations EMNLP 2021 Timo Schick, Hinrich Schütze

To obtain high-quality sentence embeddings from pretrained language models (PLMs), they must either be augmented with additional pretraining objectives or finetuned on a large set of labeled text pairs.

Semantic Textual Similarity Sentence +1

Static Embeddings as Efficient Knowledge Bases?

1 code implementation NAACL 2021 Philipp Dufter, Nora Kassner, Hinrich Schütze

Recent research investigates factual knowledge stored in large pretrained language models (PLMs).

Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP

3 code implementations28 Feb 2021 Timo Schick, Sahana Udupa, Hinrich Schütze

In this paper, we first demonstrate a surprising finding: pretrained language models recognize, to a considerable degree, their undesirable biases and the toxicity of the content they produce.

Language Modelling

Language Models for Lexical Inference in Context

1 code implementation EACL 2021 Martin Schmitt, Hinrich Schütze

Lexical inference in context (LIiC) is the task of recognizing textual entailment between two very similar sentences, i. e., sentences that only differ in one expression.

Few-Shot NLI Natural Language Inference

Improving Scene Graph Classification by Exploiting Knowledge from Texts

no code implementations9 Feb 2021 Sahand Sharifzadeh, Sina Moayed Baharlou, Martin Schmitt, Hinrich Schütze, Volker Tresp

We show that by fine-tuning the classification pipeline with the extracted knowledge from texts, we can achieve ~8x more accurate results in scene graph classification, ~3x in object classification, and ~1. 5x in predicate classification, compared to the supervised baselines with only 1% of the annotated images.

General Classification Graph Classification +7

Does He Wink or Does He Nod? A Challenging Benchmark for Evaluating Word Understanding of Language Models

no code implementations6 Feb 2021 Lutfi Kerem Senel, Hinrich Schütze

Recent progress in pretraining language models on large corpora has resulted in large performance gains on many NLP tasks.

Language Modelling

Measuring and Improving Consistency in Pretrained Language Models

1 code implementation1 Feb 2021 Yanai Elazar, Nora Kassner, Shauli Ravfogel, Abhilasha Ravichander, Eduard Hovy, Hinrich Schütze, Yoav Goldberg

In this paper we study the question: Are Pretrained Language Models (PLMs) consistent with respect to factual knowledge?

A Closer Look at Few-Shot Crosslingual Transfer: The Choice of Shots Matters

no code implementations ACL 2021 Mengjie Zhao, Yi Zhu, Ehsan Shareghi, Ivan Vulić, Roi Reichart, Anna Korhonen, Hinrich Schütze

Few-shot crosslingual transfer has been shown to outperform its zero-shot counterpart with pretrained encoders like multilingual BERT.

Few-Shot Learning

Few-Shot Text Generation with Pattern-Exploiting Training

2 code implementations22 Dec 2020 Timo Schick, Hinrich Schütze

Providing pretrained language models with simple task descriptions in natural language enables them to solve some tasks in a fully unsupervised fashion.

Headline Generation text-classification +2

Subword Sampling for Low Resource Word Alignment

no code implementations21 Dec 2020 Ehsaneddin Asgari, Masoud Jalili Sabet, Philipp Dufter, Christopher Ringlstetter, Hinrich Schütze

This method's hypothesis is that the aggregation of different granularities of text for certain language pairs can help word-level alignment.

Bayesian Optimization Machine Translation +1

Automatically Identifying Words That Can Serve as Labels for Few-Shot Text Classification

2 code implementations COLING 2020 Timo Schick, Helmut Schmid, Hinrich Schütze

A recent approach for few-shot text classification is to convert textual inputs to cloze questions that contain some form of task description, process them with a pretrained language model and map the predicted words to labels.

Few-Shot Text Classification General Classification +3

Dynamic Contextualized Word Embeddings

1 code implementation ACL 2021 Valentin Hofmann, Janet B. Pierrehumbert, Hinrich Schütze

Static word embeddings that represent words by a single vector cannot capture the variability of word meaning in different linguistic and extralinguistic contexts.

Language Modelling Word Embeddings

It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners

5 code implementations NAACL 2021 Timo Schick, Hinrich Schütze

When scaled to hundreds of billions of parameters, pretrained language models such as GPT-3 (Brown et al., 2020) achieve remarkable few-shot performance.

Natural Language Understanding

Automatic Domain Adaptation Outperforms Manual Domain Adaptation for Predicting Financial Outcomes

no code implementations ACL 2019 Marina Sedinkina, Nikolas Breitkopf, Hinrich Schütze

In our experiments, we demonstrate that the automatically adapted sentiment dictionary outperforms the previous state of the art in predicting the financial outcomes excess return and volatility.

Domain Adaptation

Neural Topic Modeling with Continual Lifelong Learning

1 code implementation ICML 2020 Pankaj Gupta, Yatin Chaudhary, Thomas Runkler, Hinrich Schütze

To address the problem, we propose a lifelong learning framework for neural topic modeling that can continuously process streams of document collections, accumulate topics and guide future topic modeling tasks by knowledge transfer from several sources to better deal with the sparse data.

Data Augmentation Information Retrieval +2

Explainable and Discourse Topic-aware Neural Language Understanding

1 code implementation ICML 2020 Yatin Chaudhary, Hinrich Schütze, Pankaj Gupta

Marrying topic models and language models exposes language understanding to a broader source of document-level context beyond sentences via topics.

Document Classification Language Modelling +5

Unsupervised Embedding-based Detection of Lexical Semantic Changes

no code implementations16 May 2020 Ehsaneddin Asgari, Christoph Ringlstetter, Hinrich Schütze

This paper describes EmbLexChange, a system introduced by the "Life-Language" team for SemEval-2020 Task 1, on unsupervised detection of lexical-semantic changes.

Identifying Necessary Elements for BERT's Multilinguality

1 code implementation1 May 2020 Philipp Dufter, Hinrich Schütze

We aim to identify architectural properties of BERT and linguistic properties of languages that are necessary for BERT to become multilingual.

Masking as an Efficient Alternative to Finetuning for Pretrained Language Models

no code implementations EMNLP 2020 Mengjie Zhao, Tao Lin, Fei Mi, Martin Jaggi, Hinrich Schütze

We present an efficient method of utilizing pretrained language models, where we learn selective binary masks for pretrained weights in lieu of modifying them through finetuning.

Quantifying the Contextualization of Word Representations with Semantic Class Probing

no code implementations Findings of the Association for Computational Linguistics 2020 Mengjie Zhao, Philipp Dufter, Yadollah Yaghoobzadeh, Hinrich Schütze

Pretrained language models have achieved a new state of the art on many NLP tasks, but there are still many open questions about how and why they work so well.

SimAlign: High Quality Word Alignments without Parallel Training Data using Static and Contextualized Embeddings

3 code implementations Findings of the Association for Computational Linguistics 2020 Masoud Jalili Sabet, Philipp Dufter, François Yvon, Hinrich Schütze

We find that alignments created from embeddings are superior for four and comparable for two language pairs compared to those produced by traditional statistical aligners, even with abundant parallel data; e. g., contextualized embeddings achieve a word alignment F1 for English-German that is 5 percentage points higher than eflomal, a high-quality statistical aligner, trained on 100k parallel sentences.

Machine Translation Multilingual Word Embeddings +3

Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference

6 code implementations21 Jan 2020 Timo Schick, Hinrich Schütze

Some NLP tasks can be solved in a fully unsupervised fashion by providing a pretrained language model with "task descriptions" in natural language (e. g., Radford et al., 2019).

Few-Shot Text Classification General Classification +3

Multipurpose Intelligent Process Automation via Conversational Assistant

no code implementations7 Jan 2020 Alena Moiseeva, Dietrich Trautmann, Michael Heimann, Hinrich Schütze

Such intelligent agents can assist the user by answering specific questions and executing routine tasks that are ordinarily performed in a natural language (i. e., customer support).

Transfer Learning

Extending Machine Language Models toward Human-Level Language Understanding

no code implementations12 Dec 2019 James L. McClelland, Felix Hill, Maja Rudolph, Jason Baldridge, Hinrich Schütze

We take language to be a part of a system for understanding and communicating about situations.

Morphological Segmentation Inside-Out

no code implementations EMNLP 2016 Ryan Cotterell, Arun Kumar, Hinrich Schütze

Morphological segmentation has traditionally been modeled with non-hierarchical models, which yield flat segmentations as output.

Morphological Analysis Segmentation

E-BERT: Efficient-Yet-Effective Entity Embeddings for BERT

1 code implementation Findings of the Association for Computational Linguistics 2020 Nina Poerner, Ulli Waltinger, Hinrich Schütze

We present a novel way of injecting factual knowledge about entities into the pretrained BERT model (Devlin et al., 2019): We align Wikipedia2Vec entity vectors (Yamada et al., 2016) with BERT's native wordpiece vector space and use the aligned entity vectors as if they were wordpiece vectors.

Entity Embeddings Entity Linking +3

Sentence Meta-Embeddings for Unsupervised Semantic Textual Similarity

no code implementations ACL 2020 Nina Poerner, Ulli Waltinger, Hinrich Schütze

We address the task of unsupervised Semantic Textual Similarity (STS) by ensembling diverse pre-trained sentence encoders into sentence meta-embeddings.

Dimensionality Reduction Semantic Textual Similarity +2

Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Fly

2 code implementations ACL 2020 Nora Kassner, Hinrich Schütze

We find that PLMs do not distinguish between negated ("Birds cannot [MASK]") and non-negated ("Birds can [MASK]") cloze questions.

Language Modelling Negation +1

BERTRAM: Improved Word Embeddings Have Big Impact on Contextualized Model Performance

1 code implementation ACL 2020 Timo Schick, Hinrich Schütze

In this work, we transfer this idea to pretrained language models: We introduce BERTRAM, a powerful architecture based on BERT that is capable of inferring high-quality embeddings for rare words that are suitable as input representations for deep language models.

Language Modelling Word Embeddings

Type-aware Convolutional Neural Networks for Slot Filling

no code implementations1 Oct 2019 Heike Adel, Hinrich Schütze

In particular, we explore different ways of integrating the named entity types of the relation arguments into a neural network for relation classification, including a joint training and a structured prediction approach.

coreference-resolution General Classification +6

Generating Multi-Sentence Abstractive Summaries of Interleaved Texts

no code implementations25 Sep 2019 Sanjeev Kumar Karn, Francine Chen, Yan-Ying Chen, Ulli Waltinger, Hinrich Schütze

The interleaved posts are encoded hierarchically, i. e., word-to-word (words in a post) followed by post-to-post (posts in a channel).

Disentanglement Sentence

Multi-source Multi-view Transfer Learning in Neural Topic Modeling with Pretrained Topic and Word Embeddings

no code implementations25 Sep 2019 Pankaj Gupta, Yatin Chaudhary, Hinrich Schütze

Though word embeddings and topics are complementary representations, several past works have only used pretrained word embeddings in (neural) topic modeling to address data sparsity problem in short text or small collection of documents.

Information Retrieval Retrieval +2

Multi-view and Multi-source Transfers in Neural Topic Modeling with Pretrained Topic and Word Embeddings

no code implementations14 Sep 2019 Pankaj Gupta, Yatin Chaudhary, Hinrich Schütze

Though word embeddings and topics are complementary representations, several past works have only used pre-trained word embeddings in (neural) topic modeling to address data sparsity problem in short text or small collection of documents.

Information Retrieval Retrieval +2

Neural Architectures for Fine-Grained Propaganda Detection in News

no code implementations WS 2019 Pankaj Gupta, Khushbu Saxena, Usama Yaseen, Thomas Runkler, Hinrich Schütze

To address the tasks of sentence (SLC) and fragment level (FLC) propaganda detection, we explore different neural architectures (e. g., CNN, LSTM-CRF and BERT) and extract linguistic (e. g., part-of-speech, named entity, readability, sentiment, emotion, etc.

Propaganda detection Sentence

A Hierarchical Decoder with Three-level Hierarchical Attention to Generate Abstractive Summaries of Interleaved Texts

no code implementations5 Jun 2019 Sanjeev Kumar Karn, Francine Chen, Yan-Ying Chen, Ulli Waltinger, Hinrich Schütze

Interleaved texts, where posts belonging to different threads occur in one sequence, are a common occurrence, e. g., online chat conversations.

SherLIiC: A Typed Event-Focused Lexical Inference Benchmark for Evaluating Natural Language Inference

1 code implementation ACL 2019 Martin Schmitt, Hinrich Schütze

We present SherLIiC, a testbed for lexical inference in context (LIiC), consisting of 3985 manually annotated inference rule candidates (InfCands), accompanied by (i) ~960k unlabeled InfCands, and (ii) ~190k typed textual relations between Freebase entities extracted from the large entity-linked corpus ClueWeb09.

Lexical Entailment Natural Language Inference

Analytical Methods for Interpretable Ultradense Word Embeddings

1 code implementation IJCNLP 2019 Philipp Dufter, Hinrich Schütze

In this work, we investigate three methods for making word spaces interpretable by rotation: Densifier (Rothe et al., 2016), linear SVMs and DensRay, a new method we propose.

Word Embeddings

Rare Words: A Major Problem for Contextualized Embeddings And How to Fix it by Attentive Mimicking

2 code implementations14 Apr 2019 Timo Schick, Hinrich Schütze

Pretraining deep neural network architectures with a language modeling objective has brought large improvements for many natural language processing tasks.

Language Modelling

Attentive Mimicking: Better Word Embeddings by Attending to Informative Contexts

1 code implementation NAACL 2019 Timo Schick, Hinrich Schütze

Learning high-quality embeddings for rare words is a hard problem because of sparse context information.

Word Embeddings

Learning Semantic Representations for Novel Words: Leveraging Both Form and Context

1 code implementation9 Nov 2018 Timo Schick, Hinrich Schütze

The general problem setting is that word embeddings are induced on an unlabeled training corpus and then a model is trained that embeds novel words into this induced embedding space.

Learning Semantic Representations Word Embeddings

Multi-Multi-View Learning: Multilingual and Multi-Representation Entity Typing

1 code implementation EMNLP 2018 Yadollah Yaghoobzadeh, Hinrich Schütze

For representation, we consider representations based on the context distribution of the entity (i. e., on its embedding), on the entity's name (i. e., on its surface form) and on its description in Wikipedia.

Entity Typing Multiview Learning +1

Neural Relation Extraction Within and Across Sentence Boundaries

1 code implementation11 Oct 2018 Pankaj Gupta, Subburam Rajaram, Hinrich Schütze, Bernt Andrassy, Thomas Runkler

iDepNN models the shortest and augmented dependency paths via recurrent and recursive neural networks to extract relationships within (intra-) and across (inter-) sentence boundaries.

Relation Relation Extraction +1

textTOvec: Deep Contextualized Neural Autoregressive Topic Models of Language with Distributed Compositional Prior

1 code implementation ICLR 2019 Pankaj Gupta, Yatin Chaudhary, Florian Buettner, Hinrich Schütze

We address two challenges of probabilistic topic modelling in order to better estimate the probability of a word in a given context, i. e., P(word|context): (1) No Language Structure in Context: Probabilistic topic models ignore word order by summarizing a given context as a "bag-of-word" and consequently the semantics of words in the context is lost.

Information Extraction Information Retrieval +4

Interpretable Textual Neuron Representations for NLP

2 code implementations WS 2018 Nina Poerner, Benjamin Roth, Hinrich Schütze

Input optimization methods, such as Google Deep Dream, create interpretable representations of neurons for computer vision DNNs.

Document Informed Neural Autoregressive Topic Models with Distributional Prior

1 code implementation15 Sep 2018 Pankaj Gupta, Yatin Chaudhary, Florian Buettner, Hinrich Schütze

Here, we extend a neural autoregressive topic model to exploit the full context information around words in a document in a language modeling fashion.

Language Modelling Retrieval +1

Neural Semi-Markov Conditional Random Fields for Robust Character-Based Part-of-Speech Tagging

no code implementations NAACL 2019 Apostolos Kemos, Heike Adel, Hinrich Schütze

Character-level models of tokens have been shown to be effective at dealing with within-token noise and out-of-vocabulary words.

Part-Of-Speech Tagging

Document Informed Neural Autoregressive Topic Models

1 code implementation11 Aug 2018 Pankaj Gupta, Florian Buettner, Hinrich Schütze

Context information around words helps in determining their actual meaning, for example "networks" used in contexts of artificial neural networks or biological neuron networks.

Language Modelling Retrieval +2

LISA: Explaining Recurrent Neural Network Judgments via Layer-wIse Semantic Accumulation and Example to Pattern Transformation

no code implementations WS 2018 Pankaj Gupta, Hinrich Schütze

Recurrent neural networks (RNNs) are temporal networks and cumulative in nature that have shown promising results in various natural language processing tasks.

Decision Making Relation Classification +2

News Article Teaser Tweets and How to Generate Them

2 code implementations NAACL 2019 Sanjeev Kumar Karn, Mark Buckley, Ulli Waltinger, Hinrich Schütze

In this work, we define the task of teaser generation and provide an evaluation benchmark and baseline systems for the process of generating teasers.

Replicated Siamese LSTM in Ticketing System for Similarity Learning and Retrieval in Asymmetric Texts

no code implementations COLING 2018 Pankaj Gupta, Bernt Andrassy, Hinrich Schütze

The task is challenging due to significant term mismatch in the query and ticket pairs of asymmetric lengths, where subject is a short text but description and solution are multi-sentence texts.

Retrieval Sentence

Fortification of Neural Morphological Segmentation Models for Polysynthetic Minimal-Resource Languages

no code implementations NAACL 2018 Katharina Kann, Manuel Mager, Ivan Meza-Ruiz, Hinrich Schütze

Morphological segmentation for polysynthetic languages is challenging, because a word may consist of many individual morphemes and training data can be extremely scarce.

Cross-Lingual Transfer Data Augmentation +1

Neural Architectures for Open-Type Relation Argument Extraction

no code implementations5 Mar 2018 Benjamin Roth, Costanza Conforti, Nina Poerner, Sanjeev Karn, Hinrich Schütze

In this work, we introduce the task of Open-Type Relation Argument Extraction (ORAE): Given a corpus, a query entity Q and a knowledge base relation (e. g.,"Q authored notable work with title X"), the model has to extract an argument of non-standard entity type (entities that cannot be extracted by a standard named entity tagger, e. g. X: the title of a book or a work of art) from the corpus.

Question Answering Relation +2

Deep Temporal-Recurrent-Replicated-Softmax for Topical Trends over Time

no code implementations NAACL 2018 Pankaj Gupta, Subburam Rajaram, Hinrich Schütze, Bernt Andrassy

We also introduce a metric (named as SPAN) to quantify the capability of dynamic topic model to capture word evolution in topics over time.

Dynamic Topic Modeling

Impact of Coreference Resolution on Slot Filling

no code implementations26 Oct 2017 Heike Adel, Hinrich Schütze

In this paper, we demonstrate the importance of coreference resolution for natural language processing on the example of the TAC Slot Filling shared task.

coreference-resolution slot-filling +1

Attentive Convolution: Equipping CNNs with RNN-style Attention Mechanisms

1 code implementation TACL 2018 Wenpeng Yin, Hinrich Schütze

We hypothesize that this is because the attention in CNNs has been mainly implemented as attentive pooling (i. e., it is applied to pooling) rather than as attentive convolution (i. e., it is integrated into convolution).

Claim Verification Natural Language Inference +3

Corpus-level Fine-grained Entity Typing

no code implementations7 Aug 2017 Yadollah Yaghoobzadeh, Heike Adel, Hinrich Schütze

This paper addresses the problem of corpus-level entity typing, i. e., inferring from a large corpus that an entity is a member of a class such as "food" or "artist".

Entity Typing Knowledge Base Completion

Unlabeled Data for Morphological Generation With Character-Based Sequence-to-Sequence Models

no code implementations WS 2017 Katharina Kann, Hinrich Schütze

We present a semi-supervised way of training a character-based encoder-decoder recurrent neural network for morphological reinflection, the task of generating one inflected word form from another.

Past, Present, Future: A Computational Investigation of the Typology of Tense in 1000 Languages

no code implementations EMNLP 2017 Ehsaneddin Asgari, Hinrich Schütze

We present SuperPivot, an analysis method for low-resource languages that occur in a superparallel corpus, i. e., in a corpus that contains an order of magnitude more languages than parallel corpora currently in use.

One-Shot Neural Cross-Lingual Transfer for Paradigm Completion

no code implementations ACL 2017 Katharina Kann, Ryan Cotterell, Hinrich Schütze

We present a novel cross-lingual transfer method for paradigm completion, the task of mapping a lemma to its inflected forms, using a neural encoder-decoder model, the state of the art for the monolingual task.

Cross-Lingual Transfer LEMMA +1

Comparative Study of CNN and RNN for Natural Language Processing

4 code implementations7 Feb 2017 Wenpeng Yin, Katharina Kann, Mo Yu, Hinrich Schütze

Deep neural networks (DNN) have revolutionized the field of natural language processing (NLP).

Position

Task-Specific Attentive Pooling of Phrase Alignments Contributes to Sentence Matching

no code implementations EACL 2017 Wenpeng Yin, Hinrich Schütze

This work studies comparatively two typical sentence matching tasks: textual entailment (TE) and answer selection (AS), observing that weaker phrase alignments are more critical in TE, while stronger phrase alignments deserve more attention in AS.

Answer Selection Natural Language Inference +2

Joint Semantic Synthesis and Morphological Analysis of the Derived Word

no code implementations TACL 2018 Ryan Cotterell, Hinrich Schütze

Since morphology obeys the principle of compositionality, the semantics of the word can be systematically derived from the meaning of its parts.

Additive models Morphological Analysis

Noise Mitigation for Neural Entity Typing and Relation Extraction

no code implementations EACL 2017 Yadollah Yaghoobzadeh, Heike Adel, Hinrich Schütze

For the second noise type, we propose ways to improve the integration of noisy entity type predictions into relation extraction.

Entity Typing Multi-Label Learning +3

Exploring Different Dimensions of Attention for Uncertainty Detection

no code implementations EACL 2017 Heike Adel, Hinrich Schütze

Neural networks with attention have proven effective for many natural language processing tasks.

Neural Multi-Source Morphological Reinflection

no code implementations EACL 2017 Katharina Kann, Ryan Cotterell, Hinrich Schütze

We explore the task of multi-source morphological reinflection, which generalizes the standard, single-source version.

LEMMA TAG

Corpus-level Fine-grained Entity Typing Using Contextual Information

no code implementations EMNLP 2015 Yadollah Yaghoobzadeh, Hinrich Schütze

This paper addresses the problem of corpus-level entity typing, i. e., inferring from a large corpus that an entity is a member of a class such as "food" or "artist".

Entity Typing Knowledge Base Completion +1

Simple Question Answering by Attentive Convolutional Neural Network

no code implementations COLING 2016 Wenpeng Yin, Mo Yu, Bing Xiang, Bo-Wen Zhou, Hinrich Schütze

In fact selection, we match the subject entity in a fact candidate with the entity mention in the question by a character-level convolutional neural network (char-CNN), and match the predicate in that fact with the question by a word-level CNN (word-CNN).

Entity Linking Fact Selection +1

Single-Model Encoder-Decoder with Explicit Morphological Representation for Reinflection

1 code implementation ACL 2016 Katharina Kann, Hinrich Schütze

Morphological reinflection is the task of generating a target form given a source form, a source tag and a target tag.

TAG

Combining Recurrent and Convolutional Neural Networks for Relation Classification

no code implementations NAACL 2016 Ngoc Thang Vu, Heike Adel, Pankaj Gupta, Hinrich Schütze

This paper investigates two different neural architectures for the task of relation classification: convolutional neural networks and recurrent neural networks.

Classification General Classification +2

Why and How to Pay Different Attention to Phrase Alignments of Different Intensities

no code implementations23 Apr 2016 Wenpeng Yin, Hinrich Schütze

We address the problems of identifying phrase alignments of flexible granularity and pooling alignments of different intensities for these tasks.

Answer Selection Natural Language Inference +3

Online Updating of Word Representations for Part-of-Speech Tagging

no code implementations EMNLP 2015 Wenpeng Yin, Tobias Schnabel, Hinrich Schütze

We propose online unsupervised domain adaptation (DA), which is performed incrementally as data comes in and is applicable when batch DA is not possible.

Online unsupervised domain adaptation Part-Of-Speech Tagging +2

Discriminative Phrase Embedding for Paraphrase Identification

no code implementations HLT 2015 Wenpeng Yin, Hinrich Schütze

This work, concerning paraphrase identification task, on one hand contributes to expanding deep learning embeddings to include continuous and discontinuous linguistic phrases.

Paraphrase Identification

Comparing Convolutional Neural Networks to Traditional Models for Slot Filling

no code implementations NAACL 2016 Heike Adel, Benjamin Roth, Hinrich Schütze

We address relation classification in the context of slot filling, the task of finding and evaluating fillers like "Steve Jobs" for the slot X in "X founded Apple".

Classification General Classification +5

ABCNN: Attention-Based Convolutional Neural Network for Modeling Sentence Pairs

8 code implementations TACL 2016 Wenpeng Yin, Hinrich Schütze, Bing Xiang, Bo-Wen Zhou

(ii) We propose three attention schemes that integrate mutual influence between sentences into CNN; thus, the representation of each sentence takes into consideration its counterpart.

Answer Selection Natural Language Inference +2

Learning Meta-Embeddings by Using Ensembles of Embedding Sets

1 code implementation18 Aug 2015 Wenpeng Yin, Hinrich Schütze

Word embeddings -- distributed representations of words -- in deep learning are beneficial for many tasks in natural language processing (NLP).

Part-Of-Speech Tagging Word Embeddings +1

Distributional Models and Deep Learning Embeddings: Combining the Best of Both Worlds

no code implementations19 Dec 2013 Irina Sergienya, Hinrich Schütze

There are two main approaches to the distributed representation of words: low-dimensional deep learning embeddings and high-dimensional distributional models, in which each dimension corresponds to a context word.

Deep Learning Embeddings for Discontinuous Linguistic Units

no code implementations18 Dec 2013 Wenpeng Yin, Hinrich Schütze

Deep learning embeddings have been successfully used for many natural language processing problems.

coreference-resolution

Cannot find the paper you are looking for? You can Submit a new open access paper.