Search Results for author:
Found 12 papers, 3 papers with code
Lookbehind-SAM: k steps back, 1 step forward
Sharpness-aware minimization (SAM) methods have gained increasing popularity by formulating the problem of minimizing both loss value and loss sharpness as a minimax objective.
Promoting Exploration in Memory-Augmented Adam using Critical Momenta
Adaptive gradient-based optimizers, particularly Adam, have left their mark in training large-scale deep learning models.
SAMSON: Sharpness-Aware Minimization Scaled by Outlier Normalization for Improving DNN Generalization and Robustness
In this work, we show that applying sharpness-aware training, by optimizing for both the loss value and loss sharpness, significantly improves robustness to noisy hardware at inference time without relying on any assumptions about the target hardware.
Improving Meta-Learning Generalization with Activation-Based Early-Stopping
Specifically, we analyze the evolution, during meta-training, of the neural activations at each hidden layer, on a small set of unlabelled support examples from a single task of the target tasks distribution, as this constitutes a minimal and justifiably accessible information from the target problem.
MemSE: Fast MSE Prediction for Noisy Memristor-Based DNN Accelerators
Memristors enable the computation of matrix-vector multiplications (MVM) in memory and, therefore, show great potential in highly increasing the energy efficiency of deep neural network (DNN) inference accelerators.
Compressing 1D Time-Channel Separable Convolutions using Sparse Random Ternary Matrices
We demonstrate that 1x1-convolutions in 1D time-channel separable convolutions may be replaced by constant, sparse random ternary matrices with weights in $\{-1, 0,+1\}$.
Evaluating Post-Training Compression in GANs using Locality-Sensitive Hashing
The analysis of the compression effects in generative adversarial networks (GANs) after training, i. e. without any fine-tuning, remains an unstudied, albeit important, topic with the increasing trend of their computation and memory requirements.
Mark-Evaluate: Assessing Language Generation using Population Estimation Methods
We propose a family of metrics to assess language generation derived from population estimation methods widely used in ecology.
Improving the Evaluation of Generative Models with Fuzzy Logic
Objective and interpretable metrics to evaluate current artificial intelligent systems are of great importance, not only to analyze the current state of such systems but also to objectively measure progress in the future.
microbatchGAN: Stimulating Diversity with Multi-Adversarial Discrimination
We propose to tackle the mode collapse problem in generative adversarial networks (GANs) by using multiple discriminators and assigning a different portion of each minibatch, called microbatch, to each discriminator.
Instant Quantization of Neural Networks using Monte Carlo Methods
Low bit-width integer weights and activations are very important for efficient inference, especially with respect to lower power consumption.
Dropout-GAN: Learning from a Dynamic Ensemble of Discriminators
We propose to incorporate adversarial dropout in generative multi-adversarial networks, by omitting or dropping out, the feedback of each discriminator in the framework with some probability at the end of each batch.
