Search Results for author:
Found 3 papers, 2 papers with code
XGrad: Boosting Gradient-Based Optimizers With Weight Prediction
the future weights to update the DNN parameters, making the gradient-based optimizer achieve better convergence and generalization compared to the original optimizer without weight prediction.
Get More at Once: Alternating Sparse Training with Gradient Correction
In this work, for the first time, we propose a novel alternating sparse training (AST) scheme to train multiple sparse sub-nets for dynamic inference without extra training cost compared to the case of training a single sparse model from scratch.
Contrastive Dual Gating: Learning Sparse Features With Contrastive Learning
Contrastive learning (or its variants) has recently become a promising direction in the self-supervised learning domain, achieving similar performance as supervised learning with minimum fine-tuning.
