Search Results for author:
Found 8 papers, 0 papers with code
Prompt Tuning for Zero-shot Compositional Learning
In order to achieve this goal, a model has to be "smart" and "knowledgeable".
Continual Diffusion: Continual Customization of Text-to-Image Diffusion with C-LoRA
We show that C-LoRA not only outperforms several baselines for our proposed setting of text-to-image continual customization, which we refer to as Continual Diffusion, but that we achieve a new state-of-the-art in the well-established rehearsal-free continual learning setting for image classification.
Numerical Optimizations for Weighted Low-rank Estimation on Language Model
However, standard SVD treats the parameters within the matrix with equal importance, which is a simple but unrealistic assumption.
Language model compression with weighted low-rank factorization
In other words, the optimization objective of SVD is not aligned with the trained model's task accuracy.
Hyperparameter-free Continuous Learning for Domain Classification in Natural Language Understanding
Most existing continual learning approaches suffer from low accuracy and performance fluctuation, especially when the distributions of old and new data are significantly different.
Lite-MDETR: A Lightweight Multi-Modal Detector
The key primitive is that Dictionary-Lookup-Transformormations (DLT) is proposed to replace Linear Transformation (LT) in multi-modal detectors where each weight in Linear Transformation (LT) is approximately factorized into a smaller dictionary, index, and coefficient.
Automatic Mixed-Precision Quantization Search of BERT
Knowledge distillation, Weight pruning, and Quantization are known to be the main directions in model compression.
DictFormer: Tiny Transformer with Shared Dictionary
DictFormer significantly reduces the redundancy in the transformer's parameters by replacing the prior transformer's parameters with compact, shared dictionary, a few unshared coefficients, and indices.
