Search Results for author:
Found 19 papers, 9 papers with code
Unsupervised Translation of German–Lower Sorbian: Exploring Training and Novel Transfer Methods on a Low-Resource Language
This paper describes the methods behind the systems submitted by the University of Groningen for the WMT 2021 Unsupervised Machine Translation task for German–Lower Sorbian (DE–DSB): a high-resource language to a low-resource one.
UDapter: Typology-based Language Adapters for Multilingual Dependency Parsing and Sequence Labeling
To address this, we propose a novel language adaptation approach by introducing contextual language adapters to a multilingual parser.
Back to Basics: Revisiting REINFORCE Style Optimization for Learning from Human Feedback in LLMs
AI alignment in the shape of Reinforcement Learning from Human Feedback (RLHF) is increasingly treated as a crucial ingredient for high performance large language models.
Aya Model: An Instruction Finetuned Open-Access Multilingual Language Model
Recent breakthroughs in large language models (LLMs) have centered around a handful of data-rich languages.
Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning
The Aya initiative also serves as a valuable case study in participatory research, involving collaborators from 119 countries.
Pushing Mixture of Experts to the Limit: Extremely Parameter Efficient MoE for Instruction Tuning
The Mixture of Experts (MoE) is a widely known neural architecture where an ensemble of specialized sub-models optimizes overall performance with a constant computational cost.
When Less is More: Investigating Data Pruning for Pretraining LLMs at Scale
In this work, we take a wider view and explore scalable estimates of data quality that can be used to systematically measure the quality of pretraining data.
Hyper-X: A Unified Hypernetwork for Multi-Task Multilingual Transfer
Massively multilingual models are promising for transfer learning across tasks and languages.
When does Parameter-Efficient Transfer Learning Work for Machine Translation?
We find that using PEFTs with a larger pre-trained model outperforms full fine-tuning with a smaller model, and for smaller training data sizes, PEFTs outperform full fine-tuning for the same pre-trained model.
Multilingual Unsupervised Neural Machine Translation with Denoising Adapters
We consider the problem of multilingual unsupervised machine translation, translating to and from languages that only have monolingual data by using auxiliary parallel language pairs.
Unsupervised Translation of German--Lower Sorbian: Exploring Training and Novel Transfer Methods on a Low-Resource Language
Lastly, we experiment with the order in which offline and online back-translation are used to train an unsupervised system, finding that using online back-translation first works better for DE$\rightarrow$DSB by 2. 76 BLEU.
On the Difficulty of Translating Free-Order Case-Marking Languages
Identifying factors that make certain languages harder to model than others is essential to reach language equality in future Natural Language Processing technologies.
From Masked Language Modeling to Translation: Non-English Auxiliary Tasks Improve Zero-shot Spoken Language Understanding
To tackle the challenge, we propose a joint learning approach, with English SLU training data and non-English auxiliary tasks from raw text, syntax and translation for transfer.
On the Effectiveness of Dataset Embeddings in Mono-lingual,Multi-lingual and Zero-shot Conditions
However, it remains unclear in which situations these dataset embeddings are most effective, because they are used in a large variety of settings, languages and tasks.
FiSSA at SemEval-2020 Task 9: Fine-tuned For Feelings
In this paper, we present our approach for sentiment classification on Spanish-English code-mixed social media data in the SemEval-2020 Task 9.
Massive Choice, Ample Tasks (MaChAmp): A Toolkit for Multi-task Learning in NLP
In this paper we present MaChAmp, a toolkit for easy fine-tuning of contextualized embeddings in multi-task settings.
UDapter: Language Adaptation for Truly Universal Dependency Parsing
The resulting parser, UDapter, outperforms strong monolingual and multilingual baselines on the majority of both high-resource and low-resource (zero-shot) languages, showing the success of the proposed adaptation approach.
A Trie-Structured Bayesian Model for Unsupervised Morphological Segmentation
Our results show that using different information sources such as neural word embeddings and letter successor variety as prior information improves morphological segmentation in a Bayesian model.
Turkish PoS Tagging by Reducing Sparsity with Morpheme Tags in Small Datasets
We learn inflectional and derivational morpheme tags in Turkish by using conditional random fields (CRF) and we employ the morpheme tags in part-of-speech (PoS) tagging by using hidden Markov models (HMMs) to mitigate sparsity.
