Search Results for author:
Found 14 papers, 4 papers with code
Zero-Space Cost Fault Tolerance for Transformer-based Language Models on ReRAM
Resistive Random Access Memory (ReRAM) has emerged as a promising platform for deep neural networks (DNNs) due to its support for parallel in-situ matrix-vector multiplication.
MaxK-GNN: Extremely Fast GPU Kernel Design for Accelerating Graph Neural Networks Training
In this paper, we present MaxK-GNN, an advanced high-performance GPU training system integrating algorithm and system innovation.
Accel-GCN: High-Performance GPU Accelerator Design for Graph Convolution Networks
Utilizing these principles, we formulated a kernel for sparse matrix multiplication (SpMM) in GCNs that employs block-level partitioning and combined warp strategy.
AutoReP: Automatic ReLU Replacement for Fast Private Network Inference
The growth of the Machine-Learning-As-A-Service (MLaaS) market has highlighted clients' data privacy and security issues.
Neurogenesis Dynamics-inspired Spiking Neural Network Training Acceleration
Experimental results show that NDSNN achieves up to 20. 52\% improvement in accuracy on Tiny-ImageNet using ResNet-19 (with a sparsity of 99\%) as compared to other SOTA methods (e. g., Lottery Ticket Hypothesis (LTH), SET-SNN, RigL-SNN).
RRNet: Towards ReLU-Reduced Neural Network for Two-party Computation Based Private Inference
The proliferation of deep learning (DL) has led to the emergence of privacy and security concerns.
Dynamic Sparse Training via Balancing the Exploration-Exploitation Trade-off
We further design an acquisition function and provide the theoretical guarantees for the proposed method and clarify its convergence property.
Efficient Traffic State Forecasting using Spatio-Temporal Network Dependencies: A Sparse Graph Neural Network Approach
We consider the problem of long-term traffic speed forecasting for a real large-scale transportation network data from the California Department of Transportation (Caltrans) Performance Measurement System (PeMS).
PolyMPCNet: Towards ReLU-free Neural Architecture Search in Two-party Computation Based Private Inference
The rapid growth and deployment of deep learning (DL) has witnessed emerging privacy and security concerns.
Towards Sparsification of Graph Neural Networks
In this paper, we utilize two state-of-the-art model compression methods (1) train and prune and (2) sparse training for the sparsification of weight layers in GNNs.
A Length Adaptive Algorithm-Hardware Co-design of Transformer on FPGA Through Sparse Attention and Dynamic Pipelining
Particularly, we develop a hardware-friendly sparse attention operator and a length-aware hardware resource scheduling algorithm.
An Automatic and Efficient BERT Pruning for Edge AI Systems
On MRPC, we obtain a 4. 6 higher score than the SOTA at the same overall pruning ratio of 0. 5.
Accelerating Framework of Transformer by Hardware Design and Model Compression Co-Optimization
Our HP can achieve higher sparsity ratio and is more flexible than other sparsity pattern.
Sparse Progressive Distillation: Resolving Overfitting under Pretrain-and-Finetune Paradigm
Conventional wisdom in pruning Transformer-based language models is that pruning reduces the model expressiveness and thus is more likely to underfit rather than overfit.
