Search Results for author:
Found 4 papers, 0 papers with code
An Isometric Stochastic Optimizer
The Adam optimizer is the standard choice in deep learning applications.
Scaling Laws for Autoregressive Generative Modeling
The optimal model size also depends on the compute budget through a power-law, with exponents that are nearly universal across all data domains.
An Empirical Study on Hyperparameters and their Interdependence for RL Generalization
Recent results in Reinforcement Learning (RL) have shown that agents with limited training environments are susceptible to a large amount of overfitting across many domains.
Semi-Supervised Learning by Label Gradient Alignment
We then formulate an optimization problem whose objective is to minimize the distance between the labeled and the unlabeled data in this space, and we solve it by gradient descent on the imputed labels.
