Search Results for author:
Found 7 papers, 1 papers with code
TEDDY: Trimming Edges with Degree-based Discrimination strategY
Since the pioneering work on the lottery ticket hypothesis for graph neural networks (GNNs) was proposed in Chen et al. (2021), the study on finding graph lottery tickets (GLT) has become one of the pivotal focus in the GNN community, inspiring researchers to discover sparser GLT while achieving comparable performance to original dense networks.
Adaptive Proximal Gradient Methods for Structured Neural Networks
We consider the training of structured neural networks where the regularizer can be non-smooth and possibly non-convex.
Cluster-Promoting Quantization with Bit-Drop for Minimizing Network Quantization Loss
Network quantization, which aims to reduce the bit-lengths of the network weights and activations, has emerged for their deployments to resource-limited devices.
Semi-Relaxed Quantization with DropBits: Training Low-Bit Neural Networks via Bitwise Regularization
As a natural extension of DropBits, we further introduce the way of learning heterogeneous quantization levels to find proper bit-length for each layer using DropBits.
A General Family of Stochastic Proximal Gradient Methods for Deep Learning
We propose a unified framework for stochastic proximal gradient descent, which we term ProxGen, that allows for arbitrary positive preconditioners and lower semi-continuous regularizers.
Semi-Relaxed Quantization with DropBits: Training Low-Bit Neural Networks via Bit-wise Regularization
As a natural extension of DropBits, we further introduce the way of learning heterogeneous quantization levels to find proper bit-length for each layer using DropBits.
Stochastic Gradient Methods with Block Diagonal Matrix Adaptation
Extensive experiments reveal that block-diagonal approaches achieve state-of-the-art results on several deep learning tasks, and can outperform adaptive diagonal methods, vanilla Sgd, as well as a modified version of full-matrix adaptation proposed very recently.
