no code implementations • 9 Jun 2022 • Damien Ferbach, Christos Tsirigotis, Gauthier Gidel, Avishek, Bose
In this paper, we generalize the SLTH to functions that preserve the action of the group $G$ -- i. e. $G$-equivariant network -- and prove, with high probability, that one can approximate any $G$-equivariant network of fixed width and depth by pruning a randomly initialized overparametrized $G$-equivariant network to a $G$-equivariant subnetwork.
1 code implementation • 1 Apr 2022 • Samuel Lavoie, Christos Tsirigotis, Max Schwarzer, Ankit Vani, Michael Noukhovitch, Kenji Kawaguchi, Aaron Courville
Simplicial Embeddings (SEM) are representations learned through self-supervised learning (SSL), wherein a representation is projected into $L$ simplices of $V$ dimensions each using a softmax operation.
2 code implementations • ICLR 2021 • Chin-wei Huang, Ricky T. Q. Chen, Christos Tsirigotis, Aaron Courville
Flow-based models are powerful tools for designing probabilistic models with tractable density.
no code implementations • ICLR 2019 • Chen Xing, Devansh Arpit, Christos Tsirigotis, Yoshua Bengio
The non-convex nature of the loss landscape of deep neural networks (DNN) lends them the intuition that over the course of training, stochastic optimization algorithms explore different regions of the loss surface by entering and escaping many local minima due to the noise induced by mini-batches.
no code implementations • 24 Feb 2018 • Chen Xing, Devansh Arpit, Christos Tsirigotis, Yoshua Bengio
Based on this and other metrics, we deduce that for most of the training update steps, SGD moves in valley like regions of the loss surface by jumping from one valley wall to another at a height above the valley floor.