Search Results for author:
Found 11 papers, 3 papers with code
Is It a Free Lunch for Removing Outliers during Pretraining?
With the growing size of large language models, the role of quantization becomes increasingly significant.
ApiQ: Finetuning of 2-Bit Quantized Large Language Model
Memory-efficient finetuning of large language models (LLMs) has recently attracted huge attention with the increasing size of LLMs, primarily due to the constraints posed by GPU memory limitations and the comparable results of these methods with full finetuning.
ITEm: Unsupervised Image-Text Embedding Learning for eCommerce
How to teach a model to learn embedding from different modalities without neglecting information from the less dominant modality is challenging.
Ask Language Model to Clean Your Noisy Translation Data
Lastly, experiments on C-MTNT showcased its effectiveness in evaluating the robustness of NMT models, highlighting the potential of advanced language models for data cleaning and emphasizing C-MTNT as a valuable resource.
Make Pre-trained Model Reversible: From Parameter to Memory Efficient Fine-Tuning
One effective way to reduce the activation memory is to apply a reversible model, so the intermediate activations are not necessary to be cached and can be recomputed.
Parameter-Efficient Fine-Tuning without Introducing New Latency
Parameter-efficient fine-tuning (PEFT) of pre-trained language models has recently demonstrated remarkable achievements, effectively matching the performance of full fine-tuning while utilizing significantly fewer trainable parameters, and consequently addressing the storage and communication constraints.
Mask More and Mask Later: Efficient Pre-training of Masked Language Models by Disentangling the [MASK] Token
We show: (1) [MASK]s can indeed be appended at a later layer, being disentangled from the word embedding; (2) The gathering of contextualized information from unmasked tokens can be conducted with a few layers.
Towards prediction of turbulent flows at high Reynolds numbers using high performance computing data and deep learning
In this paper, deep learning (DL) methods are evaluated in the context of turbulent flows.
Back-translation for Large-Scale Multilingual Machine Translation
Surprisingly, the smaller size of vocabularies perform better, and the extensive monolingual English data offers a modest improvement.
Unifying Input and Output Smoothing in Neural Machine Translation
Soft contextualized data augmentation is a recent method that replaces one-hot representation of words with soft posterior distributions of an external language model, smoothing the input of neural machine translation systems.
Multi-Agent Mutual Learning at Sentence-Level and Token-Level for Neural Machine Translation
Mutual learning, where multiple agents learn collaboratively and teach one another, has been shown to be an effective way to distill knowledge for image classification tasks.
