Search Results for author:
Found 7 papers, 0 papers with code
Instant Complexity Reduction in CNNs using Locality-Sensitive Hashing
In particular, we are able to instantly drop 46. 72% of FLOPs while only losing 1. 25% accuracy by just swapping the convolution modules in a ResNet34 on CIFAR-10 for our HASTE module.
Spectral Batch Normalization: Normalization in the Frequency Domain
However, we show experimentally that, despite the approximate additive penalty of BN, feature maps in deep neural networks (DNNs) tend to explode at the beginning of the network and that feature maps of DNNs contain large values during the whole training.
Weight Compander: A Simple Weight Reparameterization for Regularization
Large weights in deep neural networks are a sign of a more complex network that is overfitted to the training data.
Dimensionality Reduced Training by Pruning and Freezing Parts of a Deep Neural Network, a Survey
By freezing weights, the number of trainable parameters is shrunken which reduces gradient computations and the dimensionality of the model's optimization space.
Interspace Pruning: Using Adaptive Filter Representations to Improve Training of Sparse CNNs
Finally, we show that advances of IP are due to improved trainability and superior generalization ability.
COPS: Controlled Pruning Before Training Starts
The combinatorial optimization problem given by COPS is relaxed on a linear program (LP).
FreezeNet: Full Performance by Reduced Storage Costs
On the classification tasks MNIST and CIFAR-10/100 we outperform SNIP, in this setting the best reported one-shot pruning method, applied before training.
