1 code implementation • NIPS 2022 • Peng Jiang, Lihan Hu, Shihui Song
At higher sparsity, our algorithm can still match the accuracy of nonstructured sparse training in most cases, while reducing the training time by up to 5x due to the fine-grained block structures in the models.
no code implementations • 29 Sep 2021 • Shihui Song, Peng Jiang
However, we find that SCO algorithms are impractical for training GNNs on large graphs because they need to store the moving averages of the aggregated features of all nodes in the graph.
