no code implementations • 16 Sep 2023 • Shefali Garg, Zhouyuan Huo, Khe Chai Sim, Suzan Schwartz, Mason Chua, Alëna Aksënova, Tsendsuren Munkhdalai, Levi King, Darryl Wright, Zion Mengesha, Dongseong Hwang, Tara Sainath, Françoise Beaufays, Pedro Moreno Mengibar
By combining the classifier output with coarse geographic information, we can select a subset of utterances from a large corpus of untranscribed short-form queries for semi-supervised learning at scale.
no code implementations • 3 Feb 2023 • Bo Li, Dongseong Hwang, Zhouyuan Huo, Junwen Bai, Guru Prakash, Tara N. Sainath, Khe Chai Sim, Yu Zhang, Wei Han, Trevor Strohman, Francoise Beaufays
The FM encoder adapter and decoder are then finetuned to the target domain with a small amount of supervised in-domain data.
no code implementations • 4 Nov 2022 • Zhouyuan Huo, Khe Chai Sim, Bo Li, Dongseong Hwang, Tara N. Sainath, Trevor Strohman
Experimental results show that the proposed method can achieve better performance on speech recognition task than existing algorithms with fewer number of trainable parameters, less computational memory cost and faster training speed.
Automatic Speech Recognition Automatic Speech Recognition (ASR) +2
no code implementations • 13 Oct 2022 • Tara N. Sainath, Rohit Prabhavalkar, Ankur Bapna, Yu Zhang, Zhouyuan Huo, Zhehuai Chen, Bo Li, Weiran Wang, Trevor Strohman
In addition, we explore JOIST using a streaming E2E model with an order of magnitude more data, which are also novelties compared to previous works.
no code implementations • 22 Mar 2022 • Dongseong Hwang, Khe Chai Sim, Zhouyuan Huo, Trevor Strohman
State-of-the-art automatic speech recognition (ASR) systems are trained with tens of thousands of hours of labeled speech data.
Automatic Speech Recognition Automatic Speech Recognition (ASR) +2
no code implementations • 1 Oct 2021 • Zhouyuan Huo, Dongseong Hwang, Khe Chai Sim, Shefali Garg, Ananya Misra, Nikhil Siddhartha, Trevor Strohman, Françoise Beaufays
These models are typically trained on the server using transcribed speech data.
no code implementations • 1 Oct 2021 • Dongseong Hwang, Ananya Misra, Zhouyuan Huo, Nikhil Siddhartha, Shefali Garg, David Qiu, Khe Chai Sim, Trevor Strohman, Françoise Beaufays, Yanzhang He
Self- and semi-supervised learning methods have been actively investigated to reduce labeled training data or enhance the model performance.
2 code implementations • 14 Jul 2021 • Jianyu Wang, Zachary Charles, Zheng Xu, Gauri Joshi, H. Brendan McMahan, Blaise Aguera y Arcas, Maruan Al-Shedivat, Galen Andrew, Salman Avestimehr, Katharine Daly, Deepesh Data, Suhas Diggavi, Hubert Eichner, Advait Gadhikar, Zachary Garrett, Antonious M. Girgis, Filip Hanzely, Andrew Hard, Chaoyang He, Samuel Horvath, Zhouyuan Huo, Alex Ingerman, Martin Jaggi, Tara Javidi, Peter Kairouz, Satyen Kale, Sai Praneeth Karimireddy, Jakub Konecny, Sanmi Koyejo, Tian Li, Luyang Liu, Mehryar Mohri, Hang Qi, Sashank J. Reddi, Peter Richtarik, Karan Singhal, Virginia Smith, Mahdi Soltanolkotabi, Weikang Song, Ananda Theertha Suresh, Sebastian U. Stich, Ameet Talwalkar, Hongyi Wang, Blake Woodworth, Shanshan Wu, Felix X. Yu, Honglin Yuan, Manzil Zaheer, Mi Zhang, Tong Zhang, Chunxiang Zheng, Chen Zhu, Wennan Zhu
Federated learning and analytics are a distributed approach for collaboratively learning models (or statistics) from decentralized data, motivated by and designed for privacy protection.
no code implementations • NeurIPS 2021 • Zachary Charles, Zachary Garrett, Zhouyuan Huo, Sergei Shmulyian, Virginia Smith
Our work highlights a number of challenges stemming from the use of larger cohorts.
no code implementations • 14 Aug 2020 • Bin Gu, An Xu, Zhouyuan Huo, Cheng Deng, Heng Huang
To the best of our knowledge, AFSGD-VP and its SVRG and SAGA variants are the first asynchronous federated learning algorithms for vertically partitioned data.
no code implementations • 13 Aug 2020 • An Xu, Zhouyuan Huo, Heng Huang
Both our theoretical and empirical results show that our new methods can handle the "gradient mismatch" problem.
no code implementations • 25 Feb 2020 • An Xu, Zhouyuan Huo, Heng Huang
The communication of gradients is costly for training deep neural networks with multiple devices in computer vision applications.
no code implementations • 6 Feb 2020 • Zhouyuan Huo, Qian Yang, Bin Gu, Lawrence Carin. Heng Huang
Mobile crowdsensing has gained significant attention in recent years and has become a critical paradigm for emerging Internet of Things applications.
1 code implementation • 4 Feb 2020 • Zhouyuan Huo, Bin Gu, Heng Huang
Training deep neural networks using a large batch size has shown promising results and benefits many real-world applications.
8 code implementations • 10 Dec 2019 • Peter Kairouz, H. Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, Rafael G. L. D'Oliveira, Hubert Eichner, Salim El Rouayheb, David Evans, Josh Gardner, Zachary Garrett, Adrià Gascón, Badih Ghazi, Phillip B. Gibbons, Marco Gruteser, Zaid Harchaoui, Chaoyang He, Lie He, Zhouyuan Huo, Ben Hutchinson, Justin Hsu, Martin Jaggi, Tara Javidi, Gauri Joshi, Mikhail Khodak, Jakub Konečný, Aleksandra Korolova, Farinaz Koushanfar, Sanmi Koyejo, Tancrède Lepoint, Yang Liu, Prateek Mittal, Mehryar Mohri, Richard Nock, Ayfer Özgür, Rasmus Pagh, Mariana Raykova, Hang Qi, Daniel Ramage, Ramesh Raskar, Dawn Song, Weikang Song, Sebastian U. Stich, Ziteng Sun, Ananda Theertha Suresh, Florian Tramèr, Praneeth Vepakomma, Jianyu Wang, Li Xiong, Zheng Xu, Qiang Yang, Felix X. Yu, Han Yu, Sen Zhao
FL embodies the principles of focused data collection and minimization, and can mitigate many of the systemic privacy risks and costs resulting from traditional, centralized machine learning and data science approaches.
no code implementations • IJCNLP 2019 • Qian Yang, Zhouyuan Huo, Dinghan Shen, Yong Cheng, Wenlin Wang, Guoyin Wang, Lawrence Carin
Generating high-quality paraphrases is a fundamental yet challenging natural language processing task.
no code implementations • 9 Oct 2019 • Zhouyuan Huo, Heng Huang
Recently, reducing communication time between machines becomes the main focus of distributed data mining.
1 code implementation • NeurIPS 2019 • Qian Yang, Zhouyuan Huo, Wenlin Wang, Heng Huang, Lawrence Carin
Model parallelism is required if a model is too large to fit in a single computing device.
no code implementations • CVPR 2020 • An Xu, Zhouyuan Huo, Heng Huang
Training the deep convolutional neural network for computer vision problems is slow and inefficient, especially when it is large and distributed across multiple devices.
no code implementations • 16 Feb 2019 • Feihu Huang, Bin Gu, Zhouyuan Huo, Songcan Chen, Heng Huang
Proximal gradient method has been playing an important role to solve many machine learning tasks, especially for the nonsmooth problems.
no code implementations • 2 Dec 2018 • Liang Yang, Hao Jiang, Jizhong Xiao, Zhouyuan Huo
To provide a possible solution to this problem, this paper proposes a camera system with both ego-downward and third-static view to perform localization and tracking in a learning approach.
no code implementations • NeurIPS 2018 • Zhouyuan Huo, Bin Gu, Heng Huang
Training a neural network using backpropagation algorithm requires passing error gradients sequentially through the network.
no code implementations • ICML 2018 • Bin Gu, Zhouyuan Huo, Cheng Deng, Heng Huang
Asynchronous parallel stochastic gradient optimization has been playing a pivotal role to solve large-scale machine learning problems in big data applications.
3 code implementations • ICML 2018 • Zhouyuan Huo, Bin Gu, Qian Yang, Heng Huang
The backward locking in backpropagation algorithm constrains us from updating network layers in parallel and fully leveraging the computing resources.
no code implementations • 10 Nov 2017 • Zhouyuan Huo, Bin Gu, Ji Liu, Heng Huang
To the best of our knowledge, our method admits the fastest convergence rate for stochastic composition optimization: for strongly convex composition problem, our algorithm is proved to admit linear convergence; for general composition problem, our algorithm significantly improves the state-of-the-art convergence rate from $O(T^{-1/2})$ to $O((n_1+n_2)^{{2}/{3}}T^{-1})$.
no code implementations • 18 Dec 2016 • Bin Gu, De Wang, Zhouyuan Huo, Heng Huang
The theoretical results show that our inexact proximal gradient algorithms can have the same convergence rates as the ones of exact proximal gradient algorithms in the non-convex setting.
no code implementations • 5 Dec 2016 • Bin Gu, Zhouyuan Huo, Heng Huang
The convergence rate of existing asynchronous doubly stochastic zeroth order algorithms is $O(\frac{1}{\sqrt{T}})$ (also for the sequential stochastic zeroth-order optimization algorithms).
no code implementations • 29 Oct 2016 • Bin Gu, Zhouyuan Huo, Heng Huang
In this paper, we focus on a composite objective function consisting of a smooth convex function $f$ and a block separable convex function, which widely exists in machine learning and computer vision.
no code implementations • 22 Sep 2016 • Zhouyuan Huo, Bin Gu, Heng Huang
In this paper, we propose a faster method, decoupled asynchronous proximal stochastic variance reduced gradient descent method (DAP-SVRG).
no code implementations • 29 May 2016 • Zhouyuan Huo, Heng Huang
Our method does not need the dual formulation of the target problem in the optimization.
no code implementations • 12 Apr 2016 • Zhouyuan Huo, Heng Huang
We provide the first theoretical analysis on the convergence rate of the asynchronous stochastic variance reduced gradient (SVRG) descent algorithm on non-convex optimization.