1 code implementation • 29 Mar 2024 • Bowen Lei, Dongkuan Xu, Ruqi Zhang, Bani Mallick
Sparse training has emerged as a promising method for resource-efficient deep neural networks (DNNs) in real-world applications.
1 code implementation • 27 Feb 2024 • Patrick Pynadath, Riddhiman Bhattacharya, Arun Hariharan, Ruqi Zhang
Discrete distributions, particularly in high-dimensional deep models, are often highly multimodal due to inherent discontinuities.
1 code implementation • 16 Feb 2024 • Junbo Li, Zichen Miao, Qiang Qiu, Ruqi Zhang
Bayesian neural networks (BNNs) offer uncertainty quantification but come with the downside of substantially increased training and inference costs.
no code implementations • 1 Feb 2024 • Theodore Papamarkou, Maria Skoularidou, Konstantina Palla, Laurence Aitchison, Julyan Arbel, David Dunson, Maurizio Filippone, Vincent Fortuin, Philipp Hennig, Jose Miguel Hernandez Lobato, Aliaksandr Hubin, Alexander Immer, Theofanis Karaletsos, Mohammad Emtiyaz Khan, Agustinus Kristiadi, Yingzhen Li, Stephan Mandt, Christopher Nemeth, Michael A. Osborne, Tim G. J. Rudner, David Rügamer, Yee Whye Teh, Max Welling, Andrew Gordon Wilson, Ruqi Zhang
In the current landscape of deep learning research, there is a predominant emphasis on achieving high predictive accuracy in supervised tasks involving large image and language datasets.
no code implementations • 25 Oct 2023 • Ziyi Wang, Yujie Chen, Qifan Song, Ruqi Zhang
This paper investigates low-precision sampling via Stochastic Gradient Hamiltonian Monte Carlo (SGHMC) with low-precision and full-precision gradient accumulators for both strongly log-concave and non-log-concave distributions.
1 code implementation • 9 Oct 2023 • Bolian Li, Ruqi Zhang
Bayesian deep learning counts on the quality of posterior distribution estimation.
1 code implementation • ICCV 2023 • Dongyao Zhu, Bowen Lei, Jie Zhang, Yanbo Fang, Ruqi Zhang, Yiqun Xie, Dongkuan Xu
Neural networks trained on distilled data often produce over-confident output and require correction by calibration methods.
1 code implementation • 6 May 2023 • Tunazzina Islam, Ruqi Zhang, Dan Goldwasser
Climate change is the defining issue of our time, and we are at a defining moment.
1 code implementation • 10 Mar 2023 • Wanrong Zhang, Ruqi Zhang
In this paper, we study Metropolis-Hastings (MH), one of the most fundamental MCMC methods, for large-scale Bayesian inference under differential privacy.
no code implementations • 10 Mar 2023 • Bolian Li, Ruqi Zhang
Long-tailed classification poses a challenge due to its heavy imbalance in class probabilities and tail-sensitivity risks with asymmetric misprediction costs.
no code implementations • 27 Feb 2023 • Yue Xiang, Dongyao Zhu, Bowen Lei, Dongkuan Xu, Ruqi Zhang
Gradients have been exploited in proposal distributions to accelerate the convergence of Markov chain Monte Carlo algorithms on discrete distributions.
1 code implementation • 18 Feb 2023 • Bowen Lei, Ruqi Zhang, Dongkuan Xu, Bani Mallick
Previous research has shown that deep neural networks tend to be over-confident, and we find that sparse training exacerbates this problem.
1 code implementation • 9 Jan 2023 • Bowen Lei, Dongkuan Xu, Ruqi Zhang, Shuren He, Bani K. Mallick
To accelerate and stabilize the convergence of sparse training, we analyze the gradient changes and develop an adaptive gradient correction method.
1 code implementation • 12 Oct 2022 • Ruqi Zhang, Qiang Liu, Xin T. Tong
Sampling methods, as important inference and learning techniques, are typically designed for unconstrained domains.
1 code implementation • 20 Jun 2022 • Ruqi Zhang, Andrew Gordon Wilson, Christopher De Sa
While low-precision optimization has been widely used to accelerate deep learning, low-precision sampling remains largely unexplored.
1 code implementation • 20 Jun 2022 • Ruqi Zhang, Xingchao Liu, Qiang Liu
We propose discrete Langevin proposal (DLP), a simple and scalable gradient-based proposal for sampling complex high-dimensional discrete distributions.
no code implementations • 6 Jul 2020 • Ruqi Zhang, Yingzhen Li, Christopher De Sa, Sam Devlin, Cheng Zhang
Variational inference (VI) plays an essential role in approximate Bayesian inference due to its computational efficiency and broad applicability.
1 code implementation • NeurIPS 2020 • Ruqi Zhang, A. Feder Cooper, Christopher De Sa
Metropolis-Hastings (MH) is a commonly-used MCMC algorithm, but it can be intractable on large datasets due to requiring computations over the whole dataset.
1 code implementation • 29 Feb 2020 • Ruqi Zhang, A. Feder Cooper, Christopher De Sa
This improves performance, but introduces bias that can cause SGHMC to converge to the wrong distribution.
1 code implementation • NeurIPS 2019 • Ruqi Zhang, Christopher De Sa
Gibbs sampling is a Markov chain Monte Carlo method that is often used for learning and inference on graphical models.
no code implementations • pproximateinference AABI Symposium 2019 • Ruqi Zhang, Yingzhen Li, Chris De Sa, Sam Devlin, Cheng Zhang
Variational inference (VI) plays an essential role in approximate Bayesian inference due to its computational efficiency and general applicability.
4 code implementations • ICLR 2020 • Ruqi Zhang, Chunyuan Li, Jianyi Zhang, Changyou Chen, Andrew Gordon Wilson
The posteriors over neural network weights are high dimensional and multimodal.