Search Results for author:
Found 7 papers, 6 papers with code
LoRA-XS: Low-Rank Adaptation with Extremely Small Number of Parameters
The recent trend in scaling language models has led to a growing demand for parameter-efficient tuning (PEFT) methods such as LoRA (Low-Rank Adaptation).
Breaking the Language Barrier: Improving Cross-Lingual Reasoning with Structured Self-Attention
We evaluate the cross-lingual reasoning abilities of MultiLMs in two schemes: (1) where the language of the context and the question remain the same in the new languages that are tested (i. e., the reasoning is still monolingual, but the model must transfer the learned reasoning ability across languages), and (2) where the language of the context and the question is different (which we term code-switched reasoning).
Revisiting Offline Compression: Going Beyond Factorization-based Methods for Transformer Language Models
Recent transformer language models achieve outstanding results in many natural language processing (NLP) tasks.
Discovering Language-neutral Sub-networks in Multilingual Language Models
Multilingual pre-trained language models transfer remarkably well on cross-lingual downstream tasks.
AdaGrid: Adaptive Grid Search for Link Prediction Training Objective
To mitigate these limitations, we propose Adaptive Grid Search (AdaGrid), which dynamically adjusts the edge message ratio during training.
Direction is what you need: Improving Word Embedding Compression in Large Language Models
The adoption of Transformer-based models in natural language processing (NLP) has led to great success using a massive number of parameters.
Spoken dialect identification in Twitter using a multi-filter architecture
This paper presents our approach for SwissText & KONVENS 2020 shared task 2, which is a multi-stage neural model for Swiss German (GSW) identification on Twitter.
