Search Results for author:
Found 5 papers, 0 papers with code
Quantization of Large Language Models with an Overdetermined Basis
In this paper, we introduce an algorithm for data quantization based on the principles of Kashin representation.
LoTR: Low Tensor Rank Weight Adaptation
In this paper we generalize and extend an idea of low-rank adaptation (LoRA) of large language models (LLMs) based on Transformer architecture.
Run LoRA Run: Faster and Lighter LoRA Implementations
LoRA is a technique that reduces the number of trainable parameters in a neural network by introducing low-rank adapters to linear layers.
Quantization Aware Factorization for Deep Neural Network Compression
Tensor decomposition of convolutional and fully-connected layers is an effective way to reduce parameters and FLOP in neural networks.
Survey on Large Scale Neural Network Training
Modern Deep Neural Networks (DNNs) require significant memory to store weight, activations, and other intermediate tensors during training.
