Search Results for author:
Found 59 papers, 20 papers with code
Open-Source AI-based SE Tools: Opportunities and Challenges of Collaborative Software Learning
We introduce a governance framework centered on federated learning (FL), designed to foster the joint development and maintenance of open-source AI code models while safeguarding data privacy and security.
DeFT: Flash Tree-attention with IO-Awareness for Efficient Tree-search-based LLM Inference
Decoding using tree search can greatly enhance the inference quality for transformer-based Large Language Models (LLMs).
Persuading a Learning Agent
We study a repeated Bayesian persuasion problem (and more generally, any generalized principal-agent problem with complete information) where the principal does not have commitment power and the agent uses algorithms to learn to respond to the principal's signals.
Switch EMA: A Free Lunch for Better Flatness and Sharpness
Exponential Moving Average (EMA) is a widely used weight averaging (WA) regularization to learn flat optima for better generalizations without extra cost in deep neural network (DNN) optimization.
Multi-Sender Persuasion -- A Computational Perspective
The core solution concept here is the Nash equilibrium of senders' signaling policies.
Training-time Neuron Alignment through Permutation Subspace for Improving Linear Mode Connectivity and Model Fusion
In deep learning, stochastic gradient descent often yields functionally similar yet widely scattered solutions in the weight space even under the same initialization, causing barriers in the Linear Mode Connectivity (LMC) landscape.
Federated Unlearning: a Perspective of Stability and Fairness
Our key contribution lies in a comprehensive theoretical analysis of the trade-offs in FU and provides insights into data heterogeneity's impacts on FU.
PathMMU: A Massive Multimodal Expert-Level Benchmark for Understanding and Reasoning in Pathology
To address this, we introduce PathMMU, the largest and highest-quality expert-validated pathology benchmark for Large Multimodal Models (LMMs).
Learning Thresholds with Latent Values and Censored Feedback
In this paper, we investigate a problem of actively learning threshold in latent space, where the unknown reward $g(\gamma, v)$ depends on the proposed threshold $\gamma$ and latent value $v$ and it can be $only$ achieved if the threshold is lower than or equal to the unknown latent value.
On the Diversity and Realism of Distilled Dataset: An Efficient Dataset Distillation Paradigm
Contemporary machine learning requires training large neural networks on massive datasets and thus faces the challenges of high computational demands.
Towards Robust Multi-Modal Reasoning via Model Selection
The reasoning capabilities of LLM (Large Language Model) are widely acknowledged in recent research, inspiring studies on tool learning and autonomous agents.
Find Your Optimal Assignments On-the-fly: A Holistic Framework for Clustered Federated Learning
To this end, this paper presents a comprehensive investigation into current clustered FL methods and proposes a four-tier framework, namely HCFL, to encompass and extend existing approaches.
Revisiting Implicit Models: Sparsity Trade-offs Capability in Weight-tied Model for Vision Tasks
Implicit models such as Deep Equilibrium Models (DEQs) have garnered significant attention in the community for their ability to train infinite layer models with elegant solution-finding procedures and constant memory footprint.
On Pitfalls of Test-Time Adaptation
Test-Time Adaptation (TTA) has recently emerged as a promising approach for tackling the robustness challenge under distribution shifts.
Cooperative Hardware-Prompt Learning for Snapshot Compressive Imaging
In this work, we tackle this challenge by marrying prompt tuning with FL to snapshot compressive imaging for the first time and propose an federated hardware-prompt learning (FedHP) method.
Prediction with Incomplete Data under Agnostic Mask Distribution Shift
We focus on the case where the underlying joint distribution of complete features and label is invariant, but the missing pattern, i. e., mask distribution may shift agnostically between training and testing.
No Fear of Classifier Biases: Neural Collapse Inspired Federated Learning with Synthetic and Fixed Classifier
Recent advances in neural collapse have shown that the classifiers and feature prototypes under perfect training scenarios collapse into an optimal structure called simplex equiangular tight frame (ETF).
Revisiting Weighted Aggregation in Federated Learning with Neural Networks
In federated learning (FL), weighted aggregation of local models is conducted to generate a global model, and the aggregation weights are normalized (the sum of weights is 1) and proportional to the local data sizes.
Persuading a Behavioral Agent: Approximately Best Responding and Learning
We show that, under natural assumptions, (1) the sender can find a signaling scheme that guarantees itself an expected utility almost as good as its optimal utility in the classic model, no matter what approximately best-responding strategy the receiver uses; (2) on the other hand, there is no signaling scheme that gives the sender much more utility than its optimal utility in the classic model, even if the receiver uses the approximately best-responding strategy that is best for the sender.
FedRC: Tackling Diverse Distribution Shifts Challenge in Federated Learning by Robust Clustering
In this paper, we identify the learning challenges posed by the simultaneous occurrence of diverse distribution shifts and propose a clustering principle to overcome these challenges.
Decentralized Gradient Tracking with Local Steps
Gradient tracking (GT) is an algorithm designed for solving decentralized optimization problems over a network (such as training a machine learning model).
How Does Independence Help Generalization? Sample Complexity of ERM on Product Distributions
While many classical notions of learnability (e. g., PAC learnability) are distribution-free, utilizing the specific structures of an input distribution may improve learning performance.
FedBR: Improving Federated Learning on Heterogeneous Data via Local Learning Bias Reduction
As a remedy, we propose FedBR, a novel unified algorithm that reduces the local learning bias on features and classifiers to tackle these challenges.
Test-Time Robust Personalization for Federated Learning
Personalized FL additionally adapts the global model to different clients, achieving promising results on consistent local training and test distributions.
Adversarial Training for High-Stakes Reliability
We found that adversarial training increased robustness to the adversarial attacks that we trained on -- doubling the time for our contractors to find adversarial examples both with our tool (from 13 to 26 minutes) and without (from 20 to 44 minutes) -- without affecting in-distribution performance.
Learning Disentangled Behaviour Patterns for Wearable-based Human Activity Recognition
To address this issue, in this work, we proposed a Behaviour Pattern Disentanglement (BPD) framework, which can disentangle the behavior patterns from the irrelevant noises such as personal styles or environmental noises, etc.
An Improved Analysis of Gradient Tracking for Decentralized Machine Learning
We consider decentralized machine learning over a network where the training data is distributed across $n$ agents, each of which can compute stochastic model updates on their local data.
Towards Federated Learning on Time-Evolving Heterogeneous Data
Federated Learning (FL) is a learning paradigm that protects privacy by keeping client data on edge devices.
Learning by Active Forgetting for Neural Networks
This paper presents a learning model by active forgetting mechanism with artificial neural networks.
RelaySum for Decentralized Deep Learning on Heterogeneous Data
A key challenge, primarily in decentralized deep learning, remains the handling of differences between the workers' local data distributions.
Nash Convergence of Mean-Based Learning Algorithms in First Price Auctions
Specifically, the results depend on the number of bidders with the highest value: - If the number is at least three, the bidding dynamics almost surely converges to a Nash equilibrium of the auction, both in time-average and in last-iterate.
Networked Time Series Prediction with Incomplete Data via Generative Adversarial Network
An important task in such applications is to predict the future values of a NETS based on its historical values and the underlying graph.
Representation Memorization for Fast Learning New Knowledge without Forgetting
In this paper, we consider scenarios that require learning new classes or data distributions quickly and incrementally over time, as it often occurs in real-world dynamic environments.
The Multi-Modal Video Reasoning and Analyzing Competition
In this paper, we introduce the Multi-Modal Video Reasoning and Analyzing Competition (MMVRAC) workshop in conjunction with ICCV 2021.
The Optimal Size of an Epistemic Congress
We analyze the optimal size of a congress in a representative democracy.
Deep Learning for IoT
Besides, this paper presents a research on data retrieval solution to avoid hacking by adversaries in the fields of adversary machine leaning.
Consensus Control for Decentralized Deep Learning
Decentralized training of deep learning models enables on-device learning over networks, as well as efficient scaling to large compute clusters.
Quasi-Global Momentum: Accelerating Decentralized Deep Learning on Heterogeneous Data
In this paper, we investigate and identify the limitation of several decentralized optimization algorithms for different degrees of data heterogeneity.
On the Effect of Consensus in Decentralized Deep Learning
Decentralized training of deep learning models enables on-device learning over networks, as well as efficient scaling to large compute clusters.
AST-GCN: Attribute-Augmented Spatiotemporal Graph Convolutional Network for Traffic Forecasting
Traffic forecasting is a fundamental and challenging task in the field of intelligent transportation.
A Game-Theoretic Analysis of the Empirical Revenue Maximization Algorithm with Endogenous Sampling
The Empirical Revenue Maximization (ERM) is one of the most important price learning algorithms in auction design: as the literature shows it can learn approximately optimal reserve prices for revenue-maximizing auctioneers in both repeated auctions and uniform-price auctions.
XCM: An Explainable Convolutional Neural Network for Multivariate Time Series Classification
Then, we illustrate how XCM reconciles performance and explainability on a synthetic dataset and show that XCM enables a more precise identification of the regions of the input data that are important for predictions compared to the current deep learning MTS classifier also providing faithful explainability.
Learning Utilities and Equilibria in Non-Truthful Auctions
In non-truthful auctions, agents' utility for a strategy depends on the strategies of the opponents and also the prior distribution over their private types; the set of Bayes Nash equilibria generally has an intricate dependence on the prior.
On the Loss Landscape of Adversarial Training: Identifying Challenges and How to Overcome Them
We analyze the influence of adversarial training on the loss landscape of machine learning models.
Ensemble Distillation for Robust Model Fusion in Federated Learning
In most of the current training schemes the central model is refined by averaging the parameters of the server model and the updated parameters from the client side.
Extrapolation for Large-batch Training in Deep Learning
Deep learning networks are typically trained by Stochastic Gradient Descent (SGD) methods that iteratively improve the model parameters by estimating a gradient on a very small fraction of the training data.
Masking as an Efficient Alternative to Finetuning for Pretrained Language Models
We present an efficient method of utilizing pretrained language models, where we learn selective binary masks for pretrained weights in lieu of modifying them through finetuning.
Deep Collaborative Embedding for information cascade prediction
First, the existing works often assume an underlying information diffusion model, which is impractical in real world due to the complexity of information diffusion.
Overcoming Long-term Catastrophic Forgetting through Adversarial Neural Pruning and Synaptic Consolidation
It is due to two facts: first, as the model learns more tasks, the intersection of the low-error parameter subspace satisfying for these tasks becomes smaller or even does not exist; second, when the model learns a new task, the cumulative error keeps increasing as the model tries to protect the parameter configuration of previous tasks from interference.
Decentralized Deep Learning with Arbitrary Communication Compression
Decentralized training of deep learning models is a key element for enabling data privacy and on-device learning over networks, as well as for efficient scaling to large compute clusters.
Exploring Interpretable LSTM Neural Networks over Multi-Variable Data
In this paper, we explore the structure of LSTM recurrent neural networks to learn variable-wise hidden states, with the aim to capture different dynamics in multi-variable time series and distinguish the contribution of variables to the prediction.
T-GCN: A Temporal Graph ConvolutionalNetwork for Traffic Prediction
However, traffic forecasting has always been considered an open scientific issue, owing to the constraints of urban road network topological structure and the law of dynamic change with time, namely, spatial dependence and temporal dependence.
Ranked #4 on
Traffic Prediction
on SZ-Taxi
Exploring the interpretability of LSTM neural networks over multi-variable data
In learning a predictive model over multivariate time series consisting of target and exogenous variables, the forecasting performance and interpretability of the model are both essential for deployment and uncovering knowledge behind the data.
Don't Use Large Mini-Batches, Use Local SGD
Mini-batch stochastic gradient methods (SGD) are state of the art for distributed training of deep neural networks.
Multi-variable LSTM neural network for autoregressive exogenous model
In this paper, we propose multi-variable LSTM capable of accurate forecasting and variable importance interpretation for time series with exogenous variables.
An interpretable LSTM neural network for autoregressive exogenous model
In this paper, we propose an interpretable LSTM recurrent neural network, i. e., multi-variable LSTM for time series with exogenous variables.
Training DNNs with Hybrid Block Floating Point
We identify block floating point (BFP) as a promising alternative representation since it exhibits wide dynamic range and enables the majority of DNN operations to be performed with fixed-point logic.
RubyStar: A Non-Task-Oriented Mixture Model Dialog System
RubyStar is a dialog system designed to create "human-like" conversation by combining different response generation strategies.
