Search Results for author: Bao Wang

Found 40 papers, 15 papers with code

Adaptive and Implicit Regularization for Matrix Completion

1 code implementation11 Aug 2022 Zhemin Li, Tao Sun, Hongxia Wang, Bao Wang

Theoretically, we show that the adaptive regularization of \ReTwo{AIR} enhances the implicit regularization and vanishes at the end of training.

Matrix Completion

Momentum Transformer: Closing the Performance Gap Between Self-attention and Its Linearization

no code implementations1 Aug 2022 Tan Nguyen, Richard G. Baraniuk, Robert M. Kirby, Stanley J. Osher, Bao Wang

Transformers have achieved remarkable success in sequence modeling and beyond but suffer from quadratic computational and memory complexities with respect to the length of the input sequence.

Image Generation Machine Translation

Proximal Implicit ODE Solvers for Accelerating Learning Neural ODEs

no code implementations19 Apr 2022 Justin Baker, Hedi Xia, Yiwei Wang, Elena Cherkaev, Akil Narayan, Long Chen, Jack Xin, Andrea L. Bertozzi, Stanley J. Osher, Bao Wang

Learning neural ODEs often requires solving very stiff ODE systems, primarily using explicit adaptive step size ODE solvers.

Computational Efficiency

Learning POD of Complex Dynamics Using Heavy-ball Neural ODEs

1 code implementation24 Feb 2022 Justin Baker, Elena Cherkaev, Akil Narayan, Bao Wang

We compare HBNODE with other popular ROMs on several complex dynamical systems, including the von K\'{a}rm\'{a}n Street flow, the Kurganov-Petrova-Popov equation, and the one-dimensional Euler equations for fluids modeling.

glassoformer: a query-sparse transformer for post-fault power grid voltage prediction

no code implementations22 Jan 2022 Yunling Zheng, Carson Hu, Guang Lin, Meng Yue, Bao Wang, Jack Xin

Due to the sparsified queries, GLassoformer is more computationally efficient than the standard transformers.

Efficient and Reliable Overlay Networks for Decentralized Federated Learning

no code implementations12 Dec 2021 Yifan Hua, Kevin Miller, Andrea L. Bertozzi, Chen Qian, Bao Wang

As such, our proposed overlay networks accelerate convergence, improve generalization, and enhance robustness to clients failures in DFL with theoretical guarantees.

Federated Learning Generalization Bounds +2

Training Deep Neural Networks with Adaptive Momentum Inspired by the Quadratic Optimization

no code implementations18 Oct 2021 Tao Sun, Huaming Ling, Zuoqiang Shi, Dongsheng Li, Bao Wang

In this paper, to eliminate the effort for tuning the momentum-related hyperparameter, we propose a new adaptive momentum inspired by the optimal choice of the heavy ball momentum for quadratic optimization.

BIG-bench Machine Learning Image Classification +3

How Does Momentum Benefit Deep Neural Networks Architecture Design? A Few Case Studies

no code implementations13 Oct 2021 Bao Wang, Hedi Xia, Tan Nguyen, Stanley Osher

As case studies, we consider how momentum can improve the architecture design for recurrent neural networks (RNNs), neural ordinary differential equations (ODEs), and transformers.

Computational Efficiency

AIR-Net: Adaptive and Implicit Regularization Neural Network for Matrix Completion

2 code implementations12 Oct 2021 Zhemin Li, Tao Sun, Hongxia Wang, Bao Wang

Theoretically, we show that the adaptive regularization of AIR enhances the implicit regularization and vanishes at the end of training.

Matrix Completion Missing Elements

Heavy Ball Neural Ordinary Differential Equations

1 code implementation NeurIPS 2021 Hedi Xia, Vai Suliafu, Hangjie Ji, Tan M. Nguyen, Andrea L. Bertozzi, Stanley J. Osher, Bao Wang

We propose heavy ball neural ordinary differential equations (HBNODEs), leveraging the continuous limit of the classical momentum accelerated gradient descent, to improve neural ODEs (NODEs) training and inference.

Image Classification

GRAND++: Graph Neural Diffusion with A Source Term

no code implementations ICLR 2022 Matthew Thorpe, Tan Minh Nguyen, Hedi Xia, Thomas Strohmer, Andrea Bertozzi, Stanley Osher, Bao Wang

We propose GRAph Neural Diffusion with a source term (GRAND++) for graph deep learning with a limited number of labeled nodes, i. e., low-labeling rate.

Graph Learning

FMMformer: Efficient and Flexible Transformer via Decomposed Near-field and Far-field Attention

no code implementations NeurIPS 2021 Tan M. Nguyen, Vai Suliafu, Stanley J. Osher, Long Chen, Bao Wang

For instance, FMMformers achieve an average classification accuracy of $60. 74\%$ over the five Long Range Arena tasks, which is significantly better than the standard transformer's average accuracy of $58. 70\%$.

Language Modelling

Decentralized Federated Averaging

no code implementations23 Apr 2021 Tao Sun, Dongsheng Li, Bao Wang

In FedAvg, clients keep their data locally for privacy protection; a central parameter server is used to communicate between clients.

Robust Certification for Laplace Learning on Geometric Graphs

no code implementations22 Apr 2021 Matthew Thorpe, Bao Wang

Graph Laplacian (GL)-based semi-supervised learning is one of the most used approaches for classifying nodes in a graph.

Adversarial Attack Adversarial Robustness

Stability and Generalization of the Decentralized Stochastic Gradient Descent

no code implementations2 Feb 2021 Tao Sun, Dongsheng Li, Bao Wang

The stability and generalization of stochastic gradient-based methods provide valuable insights into understanding the algorithmic performance of machine learning models.

BIG-bench Machine Learning

Deep Learning with Data Privacy via Residual Perturbation

no code implementations1 Jan 2021 Wenqi Tao, Huaming Ling, Zuoqiang Shi, Bao Wang

Empirically, we show that residual perturbation outperforms the state-of-the-art DP stochastic gradient descent (DPSGD) in both membership privacy protection and maintaining the DL models' utility.

Privacy Preserving

Certifying Robustness of Graph Laplacian Based Semi-Supervised Learning

no code implementations1 Jan 2021 Matthew Thorpe, Bao Wang

Within a certain adversarial perturbation regime, we prove that GL with a $k$-nearest neighbor graph is intrinsically more robust than the $k$-nearest neighbor classifier.

Adversarial Robustness

Stochastic Gradient Descent with Nonlinear Conjugate Gradient-Style Adaptive Momentum

no code implementations3 Dec 2020 Bao Wang, Qiang Ye

In this paper, we propose a novel \emph{adaptive momentum} for improving DNNs training; this adaptive momentum, with no momentum related hyperparameter required, is motivated by the nonlinear conjugate gradient (NCG) method.

Adversarial Robustness

Deep Interactive Denoiser (DID) for X-Ray Computed Tomography

no code implementations30 Nov 2020 Ti Bai, Biling Wang, Dan Nguyen, Bao Wang, Bin Dong, Wenxiang Cong, Mannudeep K. Kalra, Steve Jiang

However, there exists two challenges regarding the DL-based denoisers: 1) a trained model typically does not generate different image candidates with different noise-resolution tradeoffs which sometimes are needed for different clinical tasks; 2) the model generalizability might be an issue when the noise level in the testing images is different from that in the training dataset.

An Integrated Approach to Produce Robust Models with High Efficiency

1 code implementation31 Aug 2020 Zhijian Li, Bao Wang, Jack Xin

To solve the problems that adversarial training jeopardizes DNNs' accuracy on clean images and the struture of sparsity, we design a trade-off loss function that helps DNNs preserve their natural accuracy and improve the channel sparsity.

Quantization Vocal Bursts Intensity Prediction

MomentumRNN: Integrating Momentum into Recurrent Neural Networks

2 code implementations NeurIPS 2020 Tan M. Nguyen, Richard G. Baraniuk, Andrea L. Bertozzi, Stanley J. Osher, Bao Wang

Designing deep neural networks is an art that often involves an expensive search over candidate architectures.

Reflections in the Sky: Joint Trajectory and Passive Beamforming Design for Secure UAV Networks with Reconfigurable Intelligent Surface

no code implementations21 May 2020 Hui Long, Ming Chen, Zhaohui Yang, Bao Wang, Zhiyang Li, Xu Yun, Mohammad Shikh-Bahaei

This paper investigates the problem of secure energy efficiency maximization for a reconfigurable intelligent surface (RIS) assisted uplink wireless communication system, where an unmanned aerial vehicle (UAV) equipped with an RIS works as a mobile relay between the base station (BS) and a group of users.

Differentially Private Federated Learning with Laplacian Smoothing

no code implementations1 May 2020 Zhicong Liang, Bao Wang, Quanquan Gu, Stanley Osher, Yuan YAO

Federated learning aims to protect data privacy by collaboratively learning a model without sharing private data among users.

Federated Learning

Sparsity Meets Robustness: Channel Pruning for the Feynman-Kac Formalism Principled Robust Deep Neural Nets

no code implementations2 Mar 2020 Thu Dinh, Bao Wang, Andrea L. Bertozzi, Stanley J. Osher

In this paper, we focus on a co-design of efficient DNN compression algorithms and sparse neural architectures for robust and accurate deep learning.

Scheduled Restart Momentum for Accelerated Stochastic Gradient Descent

1 code implementation24 Feb 2020 Bao Wang, Tan M. Nguyen, Andrea L. Bertozzi, Richard G. Baraniuk, Stanley J. Osher

Nesterov accelerated gradient (NAG) improves the convergence rate of gradient descent (GD) for convex optimization using a specially designed momentum; however, it accumulates error when an inexact gradient is used (such as in SGD), slowing convergence at best and diverging at worst.

General Classification Image Classification

Laplacian Smoothing Stochastic Gradient Markov Chain Monte Carlo

1 code implementation2 Nov 2019 Bao Wang, Difan Zou, Quanquan Gu, Stanley Osher

As an important Markov Chain Monte Carlo (MCMC) method, stochastic gradient Langevin dynamics (SGLD) algorithm has achieved great success in Bayesian learning and posterior sampling.

Graph Interpolating Activation Improves Both Natural and Robust Accuracies in Data-Efficient Deep Learning

1 code implementation16 Jul 2019 Bao Wang, Stanley J. Osher

The proposed DNN with graph interpolating activation integrates the advantages of both deep learning and manifold learning.

A Study on Graph-Structured Recurrent Neural Networks and Sparsification with Application to Epidemic Forecasting

2 code implementations13 Feb 2019 Zhijian Li, Xiyang Luo, Bao Wang, Andrea L. Bertozzi, Jack Xin

We study epidemic forecasting on real-world health data by a graph-structured recurrent neural network (GSRNN).

A Deterministic Gradient-Based Approach to Avoid Saddle Points

no code implementations21 Jan 2019 Lisa Maria Kreusser, Stanley J. Osher, Bao Wang

First-order methods such as gradient descent are usually the methods of choice for training machine learning models.

BIG-bench Machine Learning

ResNets Ensemble via the Feynman-Kac Formalism to Improve Natural and Robust Accuracies

5 code implementations NeurIPS 2019 Bao Wang, Binjie Yuan, Zuoqiang Shi, Stanley J. Osher

However, both natural and robust accuracies, in classifying clean and adversarial images, respectively, of the trained robust models are far from satisfactory.

Adversarial Attack Adversarial Defense

Mathematical Analysis of Adversarial Attacks

no code implementations15 Nov 2018 Zehao Dou, Stanley J. Osher, Bao Wang

In this paper, we analyze efficacy of the fast gradient sign method (FGSM) and the Carlini-Wagner's L2 (CW-L2) attack.

General Classification

Adversarial Defense via Data Dependent Activation Function and Total Variation Minimization

1 code implementation23 Sep 2018 Bao Wang, Alex T. Lin, Wei Zhu, Penghang Yin, Andrea L. Bertozzi, Stanley J. Osher

We improve the robustness of Deep Neural Net (DNN) to adversarial attacks by using an interpolating function as the output activation.

Adversarial Attack Adversarial Defense +1

Laplacian Smoothing Gradient Descent

1 code implementation17 Jun 2018 Stanley Osher, Bao Wang, Penghang Yin, Xiyang Luo, Farzin Barekat, Minh Pham, Alex Lin

We propose a class of very simple modifications of gradient descent and stochastic gradient descent.

Stop memorizing: A data-dependent regularization framework for intrinsic pattern learning

no code implementations ICLR 2019 Wei Zhu, Qiang Qiu, Bao Wang, Jianfeng Lu, Guillermo Sapiro, Ingrid Daubechies

Deep neural networks (DNNs) typically have enough capacity to fit random data by brute force even when conventional data-dependent regularizations focusing on the geometry of the features are imposed.

Graph-Based Deep Modeling and Real Time Forecasting of Sparse Spatio-Temporal Data

no code implementations2 Apr 2018 Bao Wang, Xiyang Luo, Fangbo Zhang, Baichuan Yuan, Andrea L. Bertozzi, P. Jeffrey Brantingham

We present a generic framework for spatio-temporal (ST) data modeling, analysis, and forecasting, with a special focus on data that is sparse in both space and time.

Deep Neural Nets with Interpolating Function as Output Activation

1 code implementation NeurIPS 2018 Bao Wang, Xiyang Luo, Zhen Li, Wei Zhu, Zuoqiang Shi, Stanley J. Osher

We replace the output layer of deep neural nets, typically the softmax function, by a novel interpolating function.

Deep Learning for Real Time Crime Forecasting

no code implementations9 Jul 2017 Bao Wang, Duo Zhang, Duanhao Zhang, P. Jeffery Brantingham, Andrea L. Bertozzi

Experiments over a half year period in Los Angeles reveal highly accurate predictive power of our models.

Crime Prediction

Feature functional theory - binding predictor (FFT-BP) for the blind prediction of binding free energies

no code implementations31 Mar 2017 Bao Wang, Zhixiong Zhao, Duc D. Nguyen, Guo-Wei Wei

The underpinning assumptions of FFT-BP are as follows: i) representability: there exists a microscopic feature vector that can uniquely characterize and distinguish one protein-ligand complex from another; ii) feature-function relationship: the macroscopic features, including binding free energy, of a complex is a functional of microscopic feature vectors; and iii) similarity: molecules with similar microscopic features have similar macroscopic features, such as binding affinity.

Cannot find the paper you are looking for? You can Submit a new open access paper.