Search Results for author:
Found 20 papers, 0 papers with code
How to Inverting the Leverage Score Distribution?
Leverage score is a fundamental problem in machine learning and theoretical computer science.
Local Convergence of Approximate Newton Method for Two Layer Nonlinear Regression
Our main results involve analyzing the convergence properties of an approximate Newton method used to minimize the regularized training loss.
Revisiting Quantum Algorithms for Linear Regressions: Quadratic Speedups without Data-Dependent Parameters
However, the running times of these algorithms depend on some quantum linear algebra-related parameters, such as $\kappa(A)$, the condition number of $A$.
The Expressibility of Polynomial based Attention Scheme
In this paper, we offer a theoretical analysis of the expressive capabilities of polynomial attention.
A Unified Scheme of ResNet and Softmax
The Hessian is shown to be positive semidefinite, and its structure is characterized as the sum of a low-rank matrix and a diagonal matrix.
A Fast Optimization View: Reformulating Single Layer Attention in LLM Based on Tensor and SVM Trick, and Solving It in Matrix Multiplication Time
$A_3$ is a matrix in $\mathbb{R}^{n \times d}$, $\mathsf{A}_{j_0} \in \mathbb{R}^{n \times d^2}$ is the $j_0$-th block of $\mathsf{A}$.
Solving Attention Kernel Regression Problem via Pre-conditioner
Given an input matrix $A\in \mathbb{R}^{n\times d}$ with $n\gg d$ and a response vector $b$, we first consider the matrix exponential of the matrix $A^\top A$ as a proxy, and we in turn design algorithms for two types of regression problems: $\min_{x\in \mathbb{R}^d}\|(A^\top A)^jx-b\|_2$ and $\min_{x\in \mathbb{R}^d}\|A(A^\top A)^jx-b\|_2$ for any positive integer $j$.
GradientCoin: A Peer-to-Peer Decentralized Large Language Models
It is likely that only two types of people would be interested in setting up a practical system for it: $\bullet$ Those who prefer to use a decentralized ChatGPT-like software.
Efficient Alternating Minimization with Applications to Weighted Low Rank Approximation
For weighted low rank approximation, this improves the runtime of [LLR16] from $n^2 k^2$ to $n^2k$.
Query Complexity of Active Learning for Function Family With Nearly Orthogonal Basis
Many machine learning algorithms require large numbers of labeled data to deliver state-of-the-art results.
Faster Robust Tensor Power Method for Arbitrary Order
Tensor decomposition is a fundamental method used in various areas to deal with high-dimensional data.
Federated Empirical Risk Minimization via Second-Order Method
Many convex optimization problems with important applications in machine learning are formulated as empirical risk minimization (ERM).
Fast and Efficient Matching Algorithm with Deadline Instances
But in \textsc{FastPostponedGreedy} algorithm, the status of each node is unknown at first.
An Iterative Algorithm for Rescaled Hyperbolic Functions Regression
LLMs have shown great promise in improving the accuracy and efficiency of these tasks, and have the potential to revolutionize the field of natural language processing (NLP) in the years to come.
Low Rank Matrix Completion via Robust Alternating Minimization in Nearly Linear Time
Moreover, our algorithm runs in time $\widetilde O(|\Omega| k)$, which is nearly linear in the time to verify the solution while preserving the sample complexity.
A Nearly-Optimal Bound for Fast Regression with $\ell_\infty$ Guarantee
One popular approach for solving such $\ell_2$ regression problem is via sketching: picking a structured random matrix $S\in \mathbb{R}^{m\times n}$ with $m\ll n$ and $SA$ can be quickly computed, solve the ``sketched'' regression problem $\arg\min_{x\in \mathbb{R}^d} \|SAx-Sb\|_2$.
A Faster $k$-means++ Algorithm
We propose a new algorithm \textsc{FastKmeans++} that only takes in $\widetilde{O}(nd + nk^2)$ time, in total.
Dynamic Maintenance of Kernel Density Estimation Data Structure: From Practice to Theory
In this work, we focus on the dynamic maintenance of KDE data structures with robustness to adversarial queries.
Sublinear Time Algorithm for Online Weighted Bipartite Matching
Online bipartite matching is a fundamental problem in online algorithms.
InstaHide's Sample Complexity When Mixing Two Private Images
On the current InstaHide challenge setup, where each InstaHide image is a mixture of two private images, we present a new algorithm to recover all the private images with a provable guarantee and optimal sample complexity.
