Search Results for author:
Found 8 papers, 4 papers with code
Quantization of Large Language Models with an Overdetermined Basis
In this paper, we introduce an algorithm for data quantization based on the principles of Kashin representation.
NAG-GS: Semi-Implicit, Accelerated and Robust Stochastic Optimizer
Classical machine learning models such as deep neural networks are usually trained by using Stochastic Gradient Descent-based (SGD) algorithms.
Memory-Efficient Backpropagation through Large Linear Layers
Also, we investigate the variance of the gradient estimate induced by the randomized matrix multiplication.
Fast Line Search for Multi-Task Learning
But, usually, line search for the step size is not the best choice due to the large computational time overhead.
A New Multi-objective Approach to Optimize Irrigation Using a Crop Simulation Model and Weather History
Finding optimal irrigation and water resources for crops is necessary to increase the efficiency of water usage.
Follow the bisector: a simple method for multi-objective optimization
This descent direction is based on the normalized gradients of the individual losses.
Stochastic gradient algorithms from ODE splitting perspective
We present a different view on stochastic optimization, which goes back to the splitting schemes for approximate solutions of ODE.
Empirical study of extreme overfitting points of neural networks
In this paper we propose a method of obtaining points of extreme overfitting - parameters of modern neural networks, at which they demonstrate close to 100 % training accuracy, simultaneously with almost zero accuracy on the test sample.
