Search Results for author:
Found 15 papers, 6 papers with code
SGD: Street View Synthesis with Gaussian Splatting and Diffusion Prior
To tackle this problem, we propose a novel approach that enhances the capacity of 3DGS by leveraging prior from a Diffusion Model along with complementary multi-modal data.
Neural Field Classifiers via Target Encoding and Classification Loss
We successfully propose a novel Neural Field Classifier (NFC) framework which formulates existing neural field methods as classification tasks rather than regression tasks.
HiCAST: Highly Customized Arbitrary Style Transfer with Adapter Enhanced Diffusion Models
In the specific, our model is constructed based on Latent Diffusion Model (LDM) and elaborately designed to absorb content and style instance as conditions of LDM.
S3IM: Stochastic Structural SIMilarity and Its Unreasonable Effectiveness for Neural Fields
Recently, Neural Radiance Field (NeRF) has shown great success in rendering novel-view images of a given scene by learning an implicit representation with only posed RGB images.
Sparse Double Descent: Where Network Pruning Aggravates Overfitting
People usually believe that network pruning not only reduces the computational cost of deep networks, but also prevents overfitting by decreasing model capacity.
Dataset Pruning: Reducing Training Data by Examining Generalization Influence
To answer these, we propose dataset pruning, an optimization-based sample selection method that can (1) examine the influence of removing a particular set of training samples on model's generalization ability with theoretical guarantee, and (2) construct the smallest subset of training data that yields strictly constrained generalization gap.
On the Power-Law Hessian Spectrums in Deep Learning
It is well-known that the Hessian of deep loss landscape matters to optimization, generalization, and even robustness of deep learning.
Adaptive Inertia: Disentangling the Effects of Adaptive Learning Rate and Momentum
Specifically, we disentangle the effects of Adaptive Learning Rate and Momentum of the Adam dynamics on saddle-point escaping and flat minima selection.
Positive-Negative Momentum: Manipulating Stochastic Gradient Noise to Improve Generalization
It is well-known that stochastic gradient noise (SGN) acts as implicit regularization for deep learning and is essentially important for both optimization and generalization of deep networks.
On the Overlooked Pitfalls of Weight Decay and How to Mitigate Them: A Gradient-Norm Perspective
Weight decay is a simple yet powerful regularization technique that has been very widely used in training of deep neural networks (DNNs).
Artificial Neural Variability for Deep Learning: On Overfitting, Noise Memorization, and Catastrophic Forgetting
Thus it motivates us to design a similar mechanism named {\it artificial neural variability} (ANV), which helps artificial neural networks learn some advantages from ``natural'' neural networks.
Stable Weight Decay Regularization
\citet{loshchilov2018decoupled} demonstrated that $L_{2}$ regularization is not identical to weight decay for adaptive gradient methods, such as Adaptive Momentum Estimation (Adam), and proposed Adam with Decoupled Weight Decay (AdamW).
Adai: Separating the Effects of Adaptive Learning Rate and Momentum Inertia
Specifically, we disentangle the effects of Adaptive Learning Rate and Momentum of the Adam dynamics on saddle-point escaping and minima selection.
A Diffusion Theory For Deep Learning Dynamics: Stochastic Gradient Descent Exponentially Favors Flat Minima
Stochastic Gradient Descent (SGD) and its variants are mainstream methods for training deep networks in practice.
A Quantum-Inspired Ensemble Method and Quantum-Inspired Forest Regressors
The contribution of this work is two-fold, a novel ensemble regression algorithm inspired by quantum mechanics and the theoretical connection between quantum interpretations and machine learning algorithms.
