Search Results for author:
Found 8 papers, 0 papers with code
How does promoting the minority fraction affect generalization? A theoretical study of the one-hidden-layer neural network on group imbalance
Despite algorithmic efforts to improve the minority group accuracy, a theoretical generalization analysis of ERM on individual groups remains elusive.
Training Nonlinear Transformers for Efficient In-Context Learning: A Theoretical Learning and Generalization Analysis
Despite the empirical success, the mechanics of how to train a Transformer to achieve ICL and the corresponding ICL capacity is mostly elusive due to the technical challenges of analyzing the nonconvex training problems resulting from the nonlinear self-attention and nonlinear activation in Transformers.
On the Convergence and Sample Complexity Analysis of Deep Q-Networks with $ε$-Greedy Exploration
This paper provides the first theoretical convergence and sample complexity analysis of the practical setting of DQNs with $\epsilon$-greedy policy.
How Can Context Help? Exploring Joint Retrieval of Passage and Personalized Context
The integration of external personalized context information into document-grounded conversational systems has significant potential business value, but has not been well-studied.
A Theoretical Understanding of Shallow Vision Transformers: Learning, Generalization, and Sample Complexity
Based on a data model characterizing both label-relevant and label-irrelevant tokens, this paper provides the first theoretical analysis of training a shallow ViT, i. e., one self-attention layer followed by a two-layer perceptron, for a classification task.
Learning and generalization of one-hidden-layer neural networks, going beyond standard Gaussian data
In addition, for the first time, this paper characterizes the impact of the input distributions on the sample complexity and the learning rate.
Generalization Guarantee of Training Graph Convolutional Networks with Graph Topology Sampling
Graph convolutional networks (GCNs) have recently achieved great empirical success in learning graph-structured data.
Learning One-hidden-layer Neural Networks on Gaussian Mixture Models with Guaranteed Generalizability
Instead of following the conventional and restrictive assumption in the literature that the input features follow the standard Gaussian distribution, this paper, for the first time, analyzes a more general and practical scenario that the input features follow a Gaussian mixture model of a finite number of Gaussian distributions of various mean and variance.
