Search Results for author:
Found 37 papers, 12 papers with code
COMQ: A Backpropagation-Free Algorithm for Post-Training Quantization
Within a fixed layer, COMQ treats all the scaling factor(s) and bit-codes as the variables of the reconstruction error.
FWin transformer for dengue prediction under climate and ocean influence
Dengue fever is one of the most deadly mosquito-born tropical infectious diseases.
A Proximal Algorithm for Network Slimming
This time-consuming, three-step process is a result of using subgradient descent to train CNNs.
Fourier-Mixed Window Attention: Accelerating Informer for Long Sequence Time-Series Forecasting
We study a fast local-global window-based attention method to accelerate Informer for long sequence time-series forecasting.
Weighted Anisotropic-Isotropic Total Variation for Poisson Denoising
Poisson noise commonly occurs in images captured by photon-limited imaging systems such as in astronomy and medicine.
Feature Affinity Assisted Knowledge Distillation and Quantization of Deep Neural Networks on Label-Free Data
In this paper, we propose a feature affinity (FA) assisted knowledge distillation (KD) method to improve quantization-aware training of deep neural networks (DNN).
Difference of Anisotropic and Isotropic TV for Segmentation under Blur and Poisson Noise
In this paper, we aim to segment an image degraded by blur and Poisson noise.
A DeepParticle method for learning and generating aggregation patterns in multi-dimensional Keller-Segel chemotaxis systems
We study a regularized interacting particle method for computing aggregation patterns and near singular solutions of a Keller-Segal (KS) chemotaxis system in two and three space dimensions, then further develop DeepParticle (DP) method to learn and generate solutions under variations of physical parameters.
Proximal Implicit ODE Solvers for Accelerating Learning Neural ODEs
Learning neural ODEs often requires solving very stiff ODE systems, primarily using explicit adaptive step size ODE solvers.
Searching Intrinsic Dimensions of Vision Transformers
It has been shown by many researchers that transformers perform as well as convolutional neural networks in many computer vision tasks.
Channel Pruning In Quantization-aware Training: An Adaptive Projection-gradient Descent-shrinkage-splitting Method
We propose an adaptive projection-gradient descent-shrinkage-splitting method (APGDSSM) to integrate penalty based channel pruning into quantization-aware training (QAT).
Enhancing Zero-Shot Many to Many Voice Conversion with Self-Attention VAE
Variational auto-encoder (VAE) is an effective neural network architecture to disentangle a speech utterance into speaker identity and linguistic content latent embeddings, then generate an utterance for a target speaker from that of a source speaker.
An Efficient Smoothing and Thresholding Image Segmentation Framework with Weighted Anisotropic-Isotropic Total Variation
In this paper, we design an efficient, multi-stage image segmentation framework that incorporates a weighted difference of anisotropic and isotropic total variation (AITV).
An integrated recurrent neural network and regression model with spatial and climatic couplings for vector-borne disease dynamics
We developed an integrated recurrent neural network and nonlinear regression spatio-temporal model for vector-borne disease evolution.
glassoformer: a query-sparse transformer for post-fault power grid voltage prediction
Due to the sparsified queries, GLassoformer is more computationally efficient than the standard transformers.
DeepParticle: learning invariant measure by a deep neural network minimizing Wasserstein distance on data generated from an interacting particle method
We introduce the so called DeepParticle method to learn and generate invariant measures of stochastic dynamical systems with physical parameters based on data computed from an interacting particle method (IPM).
Recurrence of Optimum for Training Weight and Activation Quantized Networks
Deep neural networks (DNNs) are quantized for efficient inference on resource-constrained platforms.
Learning Quantized Neural Nets by Coarse Gradient Method for Non-linear Classification
In this paper, we propose a class of STEs with certain monotonicity, and consider their applications to the training of a two-linear-layer network with quantized activation functions for non-linear multi-category classification.
A Spatial-Temporal Graph Based Hybrid Infectious Disease Model with Application to COVID-19
For edge feature, we design an RNN model to capture the neighboring effect and regularize the landscape of loss function so that local minima are effective and robust for prediction.
Improving Network Slimming with Nonconvex Regularization
Network slimming with T$\ell_1$ regularization also outperforms the latest Bayesian modification of network slimming in compressing a CNN architecture in terms of memory storage while preserving its model accuracy after channel pruning.
Lorentzian Peak Sharpening and Sparse Blind Source Separation for NMR Spectroscopy
In this paper, we introduce a preprocessing technique for blind source separation (BSS) of nonnegative and overlapped data.
An Integrated Approach to Produce Robust Models with High Efficiency
To solve the problems that adversarial training jeopardizes DNNs' accuracy on clean images and the struture of sparsity, we design a trade-off loss function that helps DNNs preserve their natural accuracy and improve the channel sparsity.
RARTS: An Efficient First-Order Relaxed Architecture Search Method
Differentiable architecture search (DARTS) is an effective method for data-driven neural network design based on solving a bilevel optimization problem.
A Recurrent Neural Network and Differential Equation Based Spatiotemporal Infectious Disease Model with Application to COVID-19
Modeling the trend of infection and real-time forecasting of cases can help decision making and control of the disease spread.
A Weighted Difference of Anisotropic and Isotropic Total Variation for Relaxed Mumford-Shah Color and Multiphase Image Segmentation
In a class of piecewise-constant image segmentation models, we propose to incorporate a weighted difference of anisotropic and isotropic total variation (AITV) to regularize the partition boundaries in an image.
Global Convergence and Geometric Characterization of Slow to Fast Weight Evolution in Neural Network Training for Classifying Linearly Non-Separable Data
In this paper, we study the dynamics of gradient descent in learning neural networks for classification problems.
$\ell_0$ Regularized Structured Sparsity Convolutional Neural Networks
Deepening and widening convolutional neural networks (CNNs) significantly increases the number of trainable weight parameters by adding more convolutional layers and feature maps per layer, respectively.
Understanding Straight-Through Estimator in Training Activation Quantized Neural Nets
We prove that if the STE is properly chosen, the expected coarse gradient correlates positively with the population gradient (not available for the training), and its negation is a descent direction for minimizing the population loss.
Learning Sparse Neural Networks via $\ell_0$ and T$\ell_1$ by a Relaxed Variable Splitting Method with Application to Multi-scale Curve Classification
We study sparsification of convolutional neural networks (CNN) by a relaxed variable splitting method of $\ell_0$ and transformed-$\ell_1$ (T$\ell_1$) penalties, with application to complex curves such as texts written in different fonts, and words written with trembling hands simulating those of Parkinson's disease patients.
A Study on Graph-Structured Recurrent Neural Networks and Sparsification with Application to Epidemic Forecasting
We study epidemic forecasting on real-world health data by a graph-structured recurrent neural network (GSRNN).
Convergence of a Relaxed Variable Splitting Coarse Gradient Descent Method for Learning Sparse Weight Binarized Activation Neural Networks
In this paper, we study the problem of coarse gradient descent (CGD) learning of a one hidden layer convolutional neural network (CNN) with binarized activation function and sparse weights.
Optimization and Control 90C26, 97R40, 68T05
AutoShuffleNet: Learning Permutation Matrices via an Exact Lipschitz Continuous Penalty in Deep Convolutional Neural Networks
In addition, we found experimentally that the standard convex relaxation of permutation matrices into stochastic matrices leads to poor performance.
An efficient model reduction method for solving viscous G-equations in incompressible cellular flows
To facilitate the algorithm design and convergence analysis, we decompose the solution of the viscous G-equation into a mean-free part and a mean part, where their evolution equations can be derived accordingly.
Numerical Analysis 65M12, 70H20, 76F25, 78M34, 80A25
Blended Coarse Gradient Descent for Full Quantization of Deep Neural Networks
We introduce the notion of coarse gradient and propose the blended coarse gradient descent (BCGD) algorithm, for training fully quantized neural networks.
BinaryRelax: A Relaxation Approach For Training Deep Neural Networks With Quantized Weights
We propose BinaryRelax, a simple two-phase algorithm, for training deep neural networks with quantized weights.
Deep Learning for Real-Time Crime Forecasting and its Ternarization
In this work, we first present a proper representation of crime data.
Quantization and Training of Low Bit-Width Convolutional Neural Networks for Object Detection
We present LBW-Net, an efficient optimization based method for quantization and training of the low bit-width convolutional neural networks (CNNs).
