Search Results for author:
Found 12 papers, 3 papers with code
FoX: Formation-aware exploration in multi-agent reinforcement learning
Recently, deep multi-agent reinforcement learning (MARL) has gained significant popularity due to its success in various cooperative multi-agent tasks.
mL-BFGS: A Momentum-based L-BFGS for Distributed Large-Scale Neural Network Optimization
Quasi-Newton methods still face significant challenges in training large-scale neural networks due to additional compute costs in the Hessian related computations and instability issues in stochastic training.
WHAT, WHEN, and HOW to Ground: Designing User Persona-Aware Conversational Agents for Engaging Dialogue
This paper presents a method for building a personalized open-domain dialogue system to address the WWH (WHAT, WHEN, and HOW) problem for natural response generation in a commercial setting, where personalized dialogue responses are heavily interleaved with casual response turns.
TimelyFL: Heterogeneity-aware Asynchronous Federated Learning with Adaptive Partial Training
However, we show empirically that this method can lead to a substantial drop in training accuracy as well as a slower convergence rate.
FedML Parrot: A Scalable Federated Learning System via Heterogeneity-aware Scheduling on Sequential and Hierarchical Training
It improves the training efficiency, remarkably relaxes the requirements on the hardware, and supports efficient large-scale FL experiments with stateful clients by: (1) sequential training clients on devices; (2) decomposing original aggregation into local and global aggregation on devices and server respectively; (3) scheduling tasks to mitigate straggler problems and enhance computing utility; (4) distributed client state manager to support various FL algorithms.
Federated Learning of Large Models at the Edge via Principal Sub-Model Training
However, the heterogeneous-client setting requires some clients to train full model, which is not aligned with the resource-constrained setting; while the latter ones break privacy promises in FL when sharing intermediate representations or labels with the server.
Partial Model Averaging in Federated Learning: Performance Guarantees and Benefits
We propose a partial model averaging framework that mitigates the model discrepancy issue in Federated Learning.
Layer-wise Adaptive Model Aggregation for Scalable Federated Learning
In Federated Learning, a common approach for aggregating local models across clients is periodic averaging of the full model parameters.
SSFL: Tackling Label Deficiency in Federated Learning via Personalized Self-Supervision
In this paper we propose self-supervised federated learning (SSFL), a unified self-supervised and personalized federated learning framework, and a series of algorithms under this framework which work towards addressing these challenges.
Toward Efficient Low-Precision Training: Data Format Optimization and Hysteresis Quantization
In this paper, we propose a method to efficiently find an optimal format without actual training of deep neural networks.
Achieving Small-Batch Accuracy with Large-Batch Scalability via Adaptive Learning Rate Adjustment
The framework performs extra epochs using the large learning rate even after the loss is flattened.
SLIM-QN: A Stochastic, Light, Momentumized Quasi-Newton Optimizer for Deep Neural Networks
SLIM-QN addresses two key barriers in existing second-order methods for large-scale DNNs: 1) the high computational cost of obtaining the Hessian matrix and its inverse in every iteration (e. g. KFAC); 2) convergence instability due to stochastic training (e. g. L-BFGS).
