1 code implementation • 22 Aug 2023 • Yonghyeon Jo, Sunwoo Lee, Junghyuk Yeom, Seungyul Han
Recently, deep multi-agent reinforcement learning (MARL) has gained significant popularity due to its success in various cooperative multi-agent tasks.
no code implementations • 25 Jul 2023 • Yue Niu, Zalan Fabian, Sunwoo Lee, Mahdi Soltanolkotabi, Salman Avestimehr
Quasi-Newton methods still face significant challenges in training large-scale neural networks due to additional compute costs in the Hessian related computations and instability issues in stochastic training.
no code implementations • 6 Jun 2023 • Deuksin Kwon, Sunwoo Lee, Ki Hyun Kim, Seojin Lee, Taeyoon Kim, Eric Davis
This paper presents a method for building a personalized open-domain dialogue system to address the WWH (WHAT, WHEN, and HOW) problem for natural response generation in a commercial setting, where personalized dialogue responses are heavily interleaved with casual response turns.
no code implementations • 14 Apr 2023 • Tuo Zhang, Lei Gao, Sunwoo Lee, Mi Zhang, Salman Avestimehr
However, we show empirically that this method can lead to a substantial drop in training accuracy as well as a slower convergence rate.
1 code implementation • 3 Mar 2023 • Zhenheng Tang, Xiaowen Chu, Ryan Yide Ran, Sunwoo Lee, Shaohuai Shi, Yonggang Zhang, Yuxin Wang, Alex Qiaozhong Liang, Salman Avestimehr, Chaoyang He
It improves the training efficiency, remarkably relaxes the requirements on the hardware, and supports efficient large-scale FL experiments with stateful clients by: (1) sequential training clients on devices; (2) decomposing original aggregation into local and global aggregation on devices and server respectively; (3) scheduling tasks to mitigate straggler problems and enhance computing utility; (4) distributed client state manager to support various FL algorithms.
1 code implementation • 28 Aug 2022 • Yue Niu, Saurav Prakash, Souvik Kundu, Sunwoo Lee, Salman Avestimehr
However, the heterogeneous-client setting requires some clients to train full model, which is not aligned with the resource-constrained setting; while the latter ones break privacy promises in FL when sharing intermediate representations or labels with the server.
no code implementations • 11 Jan 2022 • Sunwoo Lee, Anit Kumar Sahu, Chaoyang He, Salman Avestimehr
We propose a partial model averaging framework that mitigates the model discrepancy issue in Federated Learning.
no code implementations • 19 Oct 2021 • Sunwoo Lee, Tuo Zhang, Chaoyang He, Salman Avestimehr
In Federated Learning, a common approach for aggregating local models across clients is periodic averaging of the full model parameters.
no code implementations • 6 Oct 2021 • Chaoyang He, Zhengyu Yang, Erum Mushtaq, Sunwoo Lee, Mahdi Soltanolkotabi, Salman Avestimehr
In this paper we propose self-supervised federated learning (SSFL), a unified self-supervised and personalized federated learning framework, and a series of algorithms under this framework which work towards addressing these challenges.
no code implementations • 29 Sep 2021 • Sunwoo Lee, Salman Avestimehr
The framework performs extra epochs using the large learning rate even after the loss is flattened.
no code implementations • ICLR 2022 • Sunwoo Lee, Jeongwoo Park, Dongsuk Jeon
In this paper, we propose a method to efficiently find an optimal format without actual training of deep neural networks.
no code implementations • 29 Sep 2021 • Yue Niu, Zalan Fabian, Sunwoo Lee, Mahdi Soltanolkotabi, Salman Avestimehr
SLIM-QN addresses two key barriers in existing second-order methods for large-scale DNNs: 1) the high computational cost of obtaining the Hessian matrix and its inverse in every iteration (e. g. KFAC); 2) convergence instability due to stochastic training (e. g. L-BFGS).