Search Results for author:
Found 20 papers, 5 papers with code
Guided Discrete Diffusion for Electronic Health Record Generation
Electronic health records (EHRs) are a pivotal data source that enables numerous applications in computational medicine, e. g., disease progression prediction, clinical trial design, and health economics and outcomes research.
Matching the Statistical Query Lower Bound for k-sparse Parity Problems with Stochastic Gradient Descent
We then demonstrate how a trained neural network with SGD can effectively approximate this good network, solving the $k$-parity problem with small statistical errors.
Self-Play Fine-Tuning of Diffusion Models for Text-to-Image Generation
Fine-tuning Diffusion Models remains an underexplored frontier in generative artificial intelligence (GenAI), especially when compared with the remarkable progress made in fine-tuning Large Language Models (LLMs).
Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models
In this paper, we delve into the prospect of growing a strong LLM out of a weak one without the need for acquiring additional human-annotated data.
Fast Sampling via De-randomization for Discrete Diffusion Models
Despite its success in continuous spaces, discrete diffusion models, which apply to domains such as texts and natural languages, remain under-studied and often suffer from slow generation speed.
Rephrase and Respond: Let Large Language Models Ask Better Questions for Themselves
While it is widely acknowledged that the quality of a prompt, such as a question, significantly impacts the quality of the response provided by LLMs, a systematic method for crafting questions that LLMs can better comprehend is still underdeveloped.
How Many Pretraining Tasks Are Needed for In-Context Learning of Linear Regression?
Transformers pretrained on diverse tasks exhibit remarkable in-context learning (ICL) capabilities, enabling them to solve unseen tasks solely based on input contexts without adjusting model parameters.
Understanding Transferable Representation Learning and Zero-shot Transfer in CLIP
Multi-modal learning has become increasingly popular due to its ability to leverage information from different data sources (e. g., text and images) to improve the model performance.
Benign Overfitting for Two-layer ReLU Convolutional Neural Networks
We show that, under mild conditions, the neural network trained by gradient descent can achieve near-zero training loss and Bayes optimal test risk.
Finite-Sample Analysis of Learning High-Dimensional Single ReLU Neuron
On the other hand, we provide some negative results for stochastic gradient descent (SGD) for ReLU regression with symmetric Bernoulli data: if the model is well-specified, the excess risk of SGD is provably no better than that of GLM-tron ignoring constant factors, for each problem instance; and in the noiseless case, GLM-tron can achieve a small risk while SGD unavoidably suffers from a constant risk in expectation.
ISA-Net: Improved spatial attention network for PET-CT tumor segmentation
In this paper, we propose a deep learning segmentation method based on multimodal positron emission tomography-computed tomography (PET-CT), which combines the high sensitivity of PET and the precise anatomical information of CT. We design an improved spatial attention network(ISA-Net) to increase the accuracy of PET or CT in detecting tumors, which uses multi-scale convolution operation to extract feature information and can highlight the tumor region location information and suppress the non-tumor region location information.
A General Framework for Sample-Efficient Function Approximation in Reinforcement Learning
With the increasing need for handling large state and action spaces, general function approximation has become a key technique in reinforcement learning (RL).
Towards Understanding Mixture of Experts in Deep Learning
To our knowledge, this is the first result towards formally understanding the mechanism of the MoE layer for deep learning.
Benign Overfitting in Two-layer Convolutional Neural Networks
In this paper, we study the benign overfitting phenomenon in training a two-layer convolutional neural network (CNN).
Faster Perturbed Stochastic Gradient Methods for Finding Local Minima
In this paper, we propose LENA (Last stEp shriNkAge), a faster perturbed stochastic gradient framework for finding local minima.
Self-training Converts Weak Learners to Strong Learners in Mixture Models
We show that there exists a universal constant $C_{\mathrm{err}}>0$ such that if a pseudolabeler $\boldsymbol{\beta}_{\mathrm{pl}}$ can achieve classification error at most $C_{\mathrm{err}}$, then for any $\varepsilon>0$, an iterative self-training algorithm initialized at $\boldsymbol{\beta}_0 := \boldsymbol{\beta}_{\mathrm{pl}}$ using pseudolabels $\hat y = \mathrm{sgn}(\langle \boldsymbol{\beta}_t, \mathbf{x}\rangle)$ and using at most $\tilde O(d/\varepsilon^2)$ unlabeled examples suffices to learn the Bayes-optimal classifier up to $\varepsilon$ error, where $d$ is the ambient dimension.
Almost Optimal Algorithms for Two-player Zero-Sum Linear Mixture Markov Games
To assess the optimality of our algorithm, we also prove an $\tilde{\Omega}( dH\sqrt{T})$ lower bound on the regret.
A Generalized Neural Tangent Kernel Analysis for Two-layer Neural Networks
In this paper, we provide a generalized neural tangent kernel analysis and show that noisy gradient descent with weight decay can still exhibit a "kernel-like" behavior.
How Much Over-parameterization Is Sufficient to Learn Deep ReLU Networks?
A recent line of research on deep learning focuses on the extremely over-parameterized setting, and shows that when the network width is larger than a high degree polynomial of the training sample size $n$ and the inverse of the target error $\epsilon^{-1}$, deep neural networks learned by (stochastic) gradient descent enjoy nice optimization and generalization guarantees.
Stein Neural Sampler
We propose two novel samplers to generate high-quality samples from a given (un-normalized) probability density.
