Linear Warmup

Linear Warmup is a learning rate schedule where we linearly increase the learning rate from a low rate to a constant rate thereafter. This reduces volatility in the early stages of training.

Image Credit: Chengwei Zhang

Latest Papers

PAPER DATE
YOLOv4: Optimal Speed and Accuracy of Object Detection
| Alexey BochkovskiyChien-Yao WangHong-Yuan Mark Liao
2020-04-23
PointRend: Image Segmentation as Rendering
| Alexander KirillovYuxin WuKaiming HeRoss Girshick
2019-12-17
On the adequacy of untuned warmup for adaptive optimization
| Jerry MaDenis Yarats
2019-10-09
CTRL: A Conditional Transformer Language Model for Controllable Generation
| Nitish Shirish KeskarBryan McCannLav R. VarshneyCaiming XiongRichard Socher
2019-09-11
MoGA: Searching Beyond MobileNetV3
| Xiangxiang ChuBo ZhangRuijun Xu
2019-08-04
Local Relation Networks for Image Recognition
| Han HuZheng ZhangZhenda XieStephen Lin
2019-04-25
M2Det: A Single-Shot Object Detector based on Multi-Level Feature Pyramid Network
| Qijie ZhaoTao ShengYongtao WangZhi TangYing ChenLing CaiHaibin Ling
2018-11-12

Components

COMPONENT TYPE
🤖 No Components Found You can add them if they exist; e.g. Mask R-CNN uses RoIAlign

Categories