Search Results for author:
Found 89 papers, 58 papers with code
Writing System and Speaker Metadata for 2,800+ Language Varieties
We describe an open-source dataset providing metadata for about 2, 800 language varieties used in the world today.
XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalisation
However, these broad-coverage benchmarks have been mostly limited to English, and despite an increasing interest in multilingual models, a benchmark that enables the comprehensive evaluation of such methods on a diverse range of languages and tasks is still missing.
Ranked #1 on
Zero-Shot Cross-Lingual Transfer
on XTREME
(AVG metric)
Multi-Domain Multilingual Question Answering
Question answering (QA) is one of the most challenging and impactful tasks in natural language processing.
MAD-G: Multilingual Adapter Generation for Efficient Cross-Lingual Transfer
While offering (1) improved fine-tuning efficiency (by a factor of around 50 in our experiments), (2) a smaller parameter budget, and (3) increased language coverage, MAD-G remains competitive with more expensive methods for language-specific adapter training across the board.
Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning
The Aya initiative also serves as a valuable case study in participatory research, involving collaborators from 119 countries.
Gemini: A Family of Highly Capable Multimodal Models
This report introduces a new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding.
Ranked #1 on
Multi-task Language Understanding
on MMLU
(using extra training data)
Adapters: A Unified Library for Parameter-Efficient and Modular Transfer Learning
We introduce Adapters, an open-source library that unifies parameter-efficient and modular transfer learning in large language models.
Language and Task Arithmetic with Parameter-Efficient Layers for Zero-Shot Summarization
Parameter-efficient fine-tuning (PEFT) using labeled task data can significantly improve the performance of large language models (LLMs) on the downstream task.
Empowering Cross-lingual Behavioral Testing of NLP Models with Typological Features
A challenge towards developing NLP systems for the world's languages is understanding how they generalize to typological differences relevant for real-world applications.
BUFFET: Benchmarking Large Language Models for Few-shot Cross-lingual Transfer
Despite remarkable advancements in few-shot generalization in natural language processing, most models are developed and evaluated primarily in English.
mmT5: Modular Multilingual Pre-Training Solves Source Language Hallucinations
Multilingual sequence-to-sequence models perform poorly with increased language coverage and fail to consistently generate text in the correct target language in few-shot settings.
Evaluating and Modeling Attribution for Cross-Lingual Question Answering
Based on these models, we improve the attribution level of a cross-lingual question-answering system.
XTREME-UP: A User-Centric Scarce-Data Benchmark for Under-Represented Languages
We evaluate commonly used models on the benchmark.
PaLM 2 Technical Report
Through extensive evaluations on English and multilingual language, and reasoning tasks, we demonstrate that PaLM 2 has significantly improved quality on downstream tasks across different model sizes, while simultaneously exhibiting faster and more efficient inference compared to PaLM.
Ranked #1 on
Question Answering
on StrategyQA
AfriQA: Cross-lingual Open-Retrieval Question Answering for African Languages
African languages have far less in-language content available digitally, making it challenging for question answering systems to satisfy the information needs of users.
Romanization-based Large-scale Adaptation of Multilingual Language Models
In order to boost the capacity of mPLMs to deal with low-resource and unseen languages, we explore the potential of leveraging transliteration on a massive scale.
SemEval-2023 Task 12: Sentiment Analysis for African Languages (AfriSenti-SemEval)
We present the first Africentric SemEval Shared task, Sentiment Analysis for African Languages (AfriSenti-SemEval) - The dataset is available at https://github. com/afrisenti-semeval/afrisent-semeval-2023.
Modular Deep Learning
Modular deep learning has emerged as a promising solution to these challenges.
AfriSenti: A Twitter Sentiment Analysis Benchmark for African Languages
These include 75 languages with at least one million speakers each.
NusaCrowd: Open Source Initiative for Indonesian NLP Resources
We present NusaCrowd, a collaborative initiative to collect and unify existing resources for Indonesian languages, including opening access to previously non-public resources.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
QAmeleon: Multilingual QA with Only 5 Examples
The availability of large, high-quality datasets has been one of the main drivers of recent progress in question answering (QA).
TaTa: A Multilingual Table-to-Text Dataset for African Languages
To address this lack of data, we create Table-to-Text in African languages (TaTa), the first large multilingual table-to-text dataset with a focus on African languages.
MasakhaNER 2.0: Africa-centric Transfer Learning for Named Entity Recognition
African languages are spoken by over a billion people, but are underrepresented in NLP research and development.
Language Models are Multilingual Chain-of-Thought Reasoners
Finally, we show that the multilingual reasoning abilities of language models extend to other tasks such as commonsense reasoning and word-in-context semantic judgment.
Square One Bias in NLP: Towards a Multi-Dimensional Exploration of the Research Manifold
Most work targeting multilinguality, for example, considers only accuracy; most work on fairness or interpretability considers only English; and so on.
NusaX: Multilingual Parallel Sentiment Dataset for 10 Indonesian Local Languages
In this work, we focus on developing resources for languages in Indonesia.
Evaluating the Diversity, Equity and Inclusion of NLP Technology: A Case Study for Indian Languages
In order for NLP technology to be widely applicable, fair, and useful, it needs to serve a diverse set of speakers across the world's languages, be equitable, i. e., not unduly biased towards any particular language, and be inclusive of all users, particularly in low-resource settings where compute constraints are common.
Hyper-X: A Unified Hypernetwork for Multi-Task Multilingual Transfer
Massively multilingual models are promising for transfer learning across tasks and languages.
One Country, 700+ Languages: NLP Challenges for Underrepresented Languages and Dialects in Indonesia
NLP research is impeded by a lack of resources and awareness of the challenges presented by underrepresented languages and dialects.
XTREME-S: Evaluating Cross-lingual Speech Representations
Covering 102 languages from 10+ language families, 3 different domains and 4 task families, XTREME-S aims to simplify multilingual speech representation evaluation, as well as catalyze research in "universal" speech representation learning.
Expanding Pretrained Models to Thousands More Languages via Lexicon-based Adaptation
The performance of multilingual pretrained models is highly dependent on the availability of monolingual or parallel text present in a target language.
NaijaSenti: A Nigerian Twitter Sentiment Corpus for Multilingual Sentiment Analysis
We introduce the first large-scale human-annotated Twitter sentiment dataset for the four most widely spoken languages in Nigeria (Hausa, Igbo, Nigerian-Pidgin, and Yor\`ub\'a ) consisting of around 30, 000 annotated tweets per language (and 14, 000 for Nigerian-Pidgin), including a significant fraction of code-mixed tweets.
NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentation
Data augmentation is an important component in the robustness evaluation of models in natural language processing (NLP) and in enhancing the diversity of the data they are trained on.
ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning
Despite the recent success of multi-task learning and transfer learning for natural language processing (NLP), few works have systematically studied the effect of scaling up the number of tasks during pre-training.
Balancing Average and Worst-case Accuracy in Multitask Learning
When training and evaluating machine learning models on a large number of tasks, it is important to not only look at average task accuracy -- which may be biased by easy or redundant tasks -- but also worst-case accuracy (i. e. the performance on the task with the lowest accuracy).
FewNLU: Benchmarking State-of-the-Art Methods for Few-Shot Natural Language Understanding
The few-shot natural language understanding (NLU) task has attracted much recent attention.
Efficient Test Time Adapter Ensembling for Low-resource Language Varieties
Adapters are light-weight modules that allow parameter-efficient fine-tuning of pretrained models.
Charformer: Fast Character Transformers via Gradient-based Subword Tokenization
In this paper, we propose a new model inductive bias that learns a subword tokenization end-to-end as part of the model.
Ranked #3 on
Paraphrase Identification
on Quora Question Pairs
Parameter-efficient Multi-task Fine-tuning for Transformers via Shared Hypernetworks
State-of-the-art parameter-efficient fine-tuning methods rely on introducing adapter modules between the layers of a pretrained language model.
Compacter: Efficient Low-Rank Hypercomplex Adapter Layers
In this work, we propose Compacter, a method for fine-tuning large-scale language models with a better trade-off between task performance and the number of trainable parameters than prior work.
Memorisation versus Generalisation in Pre-trained Language Models
State-of-the-art pre-trained language models have been shown to memorise facts and perform well with limited amounts of training data.
IndoNLG: Benchmark and Resources for Evaluating Indonesian Natural Language Generation
Natural language generation (NLG) benchmarks provide an important avenue to measure progress and develop better NLG systems.
XTREME-R: Towards More Challenging and Nuanced Multilingual Evaluation
While a sizeable gap to human-level performance remains, improvements have been easier to achieve in some tasks than in others.
MasakhaNER: Named Entity Recognition for African Languages
We take a step towards addressing the under-representation of the African continent in NLP research by creating the first large publicly available high-quality dataset for named entity recognition (NER) in ten African languages, bringing together a variety of stakeholders.
Multi-view Subword Regularization
Multilingual pretrained representations generally rely on subword segmentation algorithms to create a shared multilingual vocabulary.
Mind the Gap: Assessing Temporal Generalization in Neural Language Models
Hence, given the compilation of ever-larger language modelling datasets, combined with the growing list of language-model-based NLP applications that require up-to-date factual knowledge about the world, we argue that now is the right time to rethink the static way in which we currently train and evaluate our language models, and develop adaptive language models that can remain up-to-date with respect to our ever-changing and non-stationary world.
Long Range Arena : A Benchmark for Efficient Transformers
Transformers do not scale very well to long sequence lengths largely because of quadratic self-attention complexity.
How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models
In this work, we provide a systematic and comprehensive empirical comparison of pretrained multilingual language models versus their monolingual counterparts with regard to their monolingual task performance.
UNKs Everywhere: Adapting Multilingual Language Models to New Scripts
The ultimate challenge is dealing with under-resourced languages not covered at all by the models and written in scripts unseen during pretraining.
Morphologically Aware Word-Level Translation
We propose a novel morphologically aware probability model for bilingual lexicon induction, which jointly models lexeme translation and inflectional morphology in a structured way.
Long Range Arena: A Benchmark for Efficient Transformers
In the recent months, a wide spectrum of efficient, fast Transformers have been proposed to tackle this problem, more often than not claiming superior or comparable model quality to vanilla Transformer models.
Ranked #18 on
Long-range modeling
on LRA
(Pathfinder metric)
Rethinking embedding coupling in pre-trained language models
We re-evaluate the standard practice of sharing weights between input and output embeddings in state-of-the-art pre-trained language models.
Ranked #1 on
Cross-Lingual NER
on NER
Cross-Lingual Natural Language Inference
Cross-Lingual NER
+4
AdapterHub: A Framework for Adapting Transformers
We propose AdapterHub, a framework that allows dynamic "stitching-in" of pre-trained adapters for different tasks and languages.
MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfer
The main goal behind state-of-the-art pre-trained multilingual models such as multilingual BERT and XLM-R is enabling and bootstrapping NLP applications in low-resource languages through zero-shot or few-shot cross-lingual transfer.
Ranked #5 on
Cross-Lingual Transfer
on XCOPA
(using extra training data)
A Call for More Rigor in Unsupervised Cross-lingual Learning
We review motivations, definition, approaches, and methodology for unsupervised cross-lingual learning and call for a more rigorous position in each of them.
AxCell: Automatic Extraction of Results from Machine Learning Papers
Tracking progress in machine learning has become increasingly difficult with the recent explosion in the number of papers.
Are All Good Word Vector Spaces Isomorphic?
Existing algorithms for aligning cross-lingual word vector spaces assume that vector spaces are approximately isomorphic.
XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization
However, these broad-coverage benchmarks have been mostly limited to English, and despite an increasing interest in multilingual models, a benchmark that enables the comprehensive evaluation of such methods on a diverse range of languages and tasks is still missing.
On the Cross-lingual Transferability of Monolingual Representations
This generalization ability has been attributed to the use of a shared subword vocabulary and joint training across multiple languages giving rise to deep multilingual abstractions.
MultiFiT: Efficient Multi-lingual Language Model Fine-tuning
Pretrained language models are promising particularly for low-resource languages as they only require unlabelled data.
Ranked #1 on
Zero-shot Cross-Lingual Document Classification
on Cross-Lingual Sentiment (CLS)- English to German - DVD
Cross-Lingual Document Classification
Document Classification
+2
What do Deep Networks Like to Read?
Recent research towards understanding neural networks probes models in a top-down manner, but is only able to identify model tendencies that are known a priori.
Don't Forget the Long Tail! A Comprehensive Analysis of Morphological Generalization in Bilingual Lexicon Induction
Human translators routinely have to translate rare inflections of words - due to the Zipfian distribution of words in a language.
Unsupervised Cross-Lingual Representation Learning
In this tutorial, we provide a comprehensive survey of the exciting recent work on cutting-edge weakly-supervised and unsupervised cross-lingual word representations.
Episodic Memory in Lifelong Language Learning
We introduce a lifelong language learning setup where a model needs to learn from a stream of text examples without any dataset identifier.
Transfer Learning in Natural Language Processing
The classic supervised machine learning paradigm is based on learning in isolation, a single predictive model for a task using a single dataset.
To Tune or Not to Tune? Adapting Pretrained Representations to Diverse Tasks
While most previous work has focused on different pretraining objectives and architectures for transfer learning, we ask how to best adapt the pretrained model to a given target task.
How to (Properly) Evaluate Cross-Lingual Word Embeddings: On Strong Baselines, Comparative Analyses, and Some Misconceptions
In this work, we make the first step towards a comprehensive evaluation of cross-lingual word embeddings.
A Hierarchical Multi-task Approach for Learning Embeddings from Semantic Tasks
The model is trained in a hierarchical fashion to introduce an inductive bias by supervising a set of low level tasks at the bottom layers of the model and more complex tasks at the top layers of the model.
Ranked #10 on
Relation Extraction
on ACE 2005
(using extra training data)
Off-the-Shelf Unsupervised NMT
We frame unsupervised machine translation (MT) in the context of multi-task learning (MTL), combining insights from both directions.
Generalizing Procrustes Analysis for Better Bilingual Dictionary Induction
Most recent approaches to bilingual dictionary induction find a linear alignment between the word vector spaces of two languages.
A Discriminative Latent-Variable Model for Bilingual Lexicon Induction
We introduce a novel discriminative latent variable model for bilingual lexicon induction.
360\mbox$^\circ$ Stance Detection
To ameliorate this, we propose 360{\mbox{$^\circ$}} Stance Detection, a tool that aggregates news with multiple perspectives on a topic.
On the Limitations of Unsupervised Bilingual Dictionary Induction
Unsupervised machine translation---i. e., not assuming any cross-lingual supervision signal, whether a dictionary, translations, or comparable corpora---seems impossible, but nevertheless, Lample et al. (2018) recently proposed a fully unsupervised machine translation (MT) model.
Strong Baselines for Neural Semi-supervised Learning under Domain Shift
In this paper, we re-evaluate classic general-purpose bootstrapping approaches in the context of neural networks under domain shifts vs. recent neural approaches and propose a novel multi-task tri-training method that reduces the time and space complexity of classic tri-training.
Ranked #3 on
Sentiment Analysis
on Multi-Domain Sentiment Dataset
360° Stance Detection
To ameliorate this, we propose 360{\deg} Stance Detection, a tool that aggregates news with multiple perspectives on a topic.
Multi-task Learning of Pairwise Sequence Classification Tasks Over Disparate Label Spaces
We combine multi-task learning and semi-supervised learning by inducing a joint embedding space between disparate label spaces and learning transfer functions between label embeddings, enabling us to jointly leverage unlabelled data and auxiliary, annotated datasets.
Universal Language Model Fine-tuning for Text Classification
Inductive transfer learning has greatly impacted computer vision, but existing approaches in NLP still require task-specific modifications and training from scratch.
Ranked #4 on
Text Classification
on AG News
Learning to select data for transfer learning with Bayesian Optimization
Domain similarity measures can be used to gauge adaptability and select suitable data for transfer learning, but existing approaches define ad hoc measures that are deemed suitable for respective tasks.
An Overview of Multi-Task Learning in Deep Neural Networks
Multi-task learning (MTL) has led to successes in many applications of machine learning, from natural language processing and speech recognition to computer vision and drug discovery.
A Survey Of Cross-lingual Word Embedding Models
Cross-lingual representations of words enable us to reason about word meaning in multilingual contexts and are a key facilitator of cross-lingual transfer when developing natural language processing models for low-resource languages.
Latent Multi-task Architecture Learning
In practice, however, MTL involves searching an enormous space of possible parameter sharing architectures to find (a) the layers or subspaces that benefit from sharing, (b) the appropriate amount of sharing, and (c) the appropriate relative weights of the different task losses.
Data Selection Strategies for Multi-Domain Sentiment Analysis
However, the selection of appropriate training data is as important as the choice of algorithm.
Knowledge Adaptation: Teaching to Adapt
Domain adaptation is crucial in many real-world applications where the distribution of the training data differs from the distribution of the test data.
Towards a continuous modeling of natural language domains
Humans continuously adapt their style and language to a variety of domains.
Character-level and Multi-channel Convolutional Neural Networks for Large-scale Authorship Attribution
Convolutional neural networks (CNNs) have demonstrated superior capability for extracting information from raw signals in computer vision.
An overview of gradient descent optimization algorithms
Gradient descent optimization algorithms, while increasingly popular, are often used as black-box optimizers, as practical explanations of their strengths and weaknesses are hard to come by.
INSIGHT-1 at SemEval-2016 Task 5: Deep Learning for Multilingual Aspect-based Sentiment Analysis
This paper describes our deep learning-based approach to multilingual aspect-based sentiment analysis as part of SemEval 2016 Task 5.
Aspect-Based Sentiment Analysis
Aspect Category Detection
+3
INSIGHT-1 at SemEval-2016 Task 4: Convolutional Neural Networks for Sentiment Classification and Quantification
This paper describes our deep learning-based approach to sentiment analysis in Twitter as part of SemEval-2016 Task 4.
A Hierarchical Model of Reviews for Aspect-based Sentiment Analysis
Opinion mining from customer reviews has become pervasive in recent years.
Aspect-Based Sentiment Analysis
Aspect-Based Sentiment Analysis (ABSA)
+3
