Search Results for author:
Found 14 papers, 5 papers with code
Surrogate Gap Minimization Improves Sharpness-Aware Training
Instead, we define a \textit{surrogate gap}, a measure equivalent to the dominant eigenvalue of Hessian at a local minimum when the radius of the neighborhood (to derive the perturbed loss) is small.
Momentum Centering and Asynchronous Update for Adaptive Gradient Methods
We demonstrate that ACProp has a convergence rate of $O(\frac{1}{\sqrt{T}})$ for the stochastic non-convex case, which matches the oracle rate and outperforms the $O(\frac{logT}{\sqrt{T}})$ rate of RMSProp and Adam.
Multiple-shooting adjoint method for whole-brain dynamic causal modeling
Furthermore, MSA uses the adjoint method for accurate gradient estimation in the ODE; since the adjoint method is generic, MSA is a generic method for both linear and non-linear systems, and does not require re-derivation of the algorithm as in EM.
MALI: A memory efficient and reverse accurate integrator for Neural ODEs
Neural ordinary differential equations (Neural ODEs) are a new family of deep-learning models with continuous depth.
Ranked #19 on
Image Generation
on ImageNet 64x64
(Bits per dim metric)
AdaBelief Optimizer: Adapting Stepsizes by theBelief in Observed Gradients
We propose AdaBelief optimizer to simultaneously achieve three goals: fast convergence as in adaptive methods, good generalization as in SGD, and training stability.
AdaBelief Optimizer: Adapting Stepsizes by the Belief in Observed Gradients
Viewing the exponential moving average (EMA) of the noisy gradient as the prediction of the gradient at the next time step, if the observed gradient greatly deviates from the prediction, we distrust the current observation and take a small step; if the observed gradient is close to the prediction, we trust it and take a large step.
Adaptive Checkpoint Adjoint Method for Gradient Estimation in Neural ODE
Neural ordinary differential equations (NODEs) have recently attracted increasing attention; however, their empirical performance on benchmark tasks (e. g. image classification) are significantly inferior to discrete-layer models.
Zero-shot Transfer Learning for Semantic Parsing
In this paper we propose to use feature transfer in a zero-shot experimental setting on the task of semantic parsing.
Sequence to Logic with Copy and Cache
Generating logical form equivalents of human language is a fresh way to employ neural architectures where long short-term memory effectively captures dependencies in both encoder and decoder units.
A New Approach to Laplacian Solvers and Flow Problems
This paper investigates the behavior of the Min-Sum message passing scheme to solve systems of linear equations in the Laplacian matrices of graphs and to compute electric flows.
Scale-free network optimization: foundations and algorithms
We propose a notion of correlation in constrained optimization that is based on the sensitivity of the optimal solution upon perturbations of the constraints.
Scatter Matrix Concordance: A Diagnostic for Regressions on Subsets of Data
Linear regression models depend directly on the design matrix and its properties.
Lossy Compression via Sparse Linear Regression: Computationally Efficient Encoding and Decoding
The proposed encoding algorithm sequentially chooses columns of the design matrix to successively approximate the source sequence.
Lossy Compression via Sparse Linear Regression: Performance under Minimum-distance Encoding
We study a new class of codes for lossy compression with the squared-error distortion criterion, designed using the statistical framework of high-dimensional linear regression.
