Learning Rate Schedules
Linear Warmup
Linear Warmup is a learning rate schedule where we linearly increase the learning rate from a low rate to a constant rate thereafter. This reduces volatility in the early stages of training.
Image Credit: Chengwei Zhang
Papers
| Paper | Code | Results | Date | Stars |
|---|
Tasks
| Task | Papers | Share |
|---|---|---|
| Language Modelling | 7 | 11.86% |
| Text Generation | 4 | 6.78% |
| Self-Supervised Learning | 3 | 5.08% |
| Object Detection | 3 | 5.08% |
| Backdoor Attack | 2 | 3.39% |
| Semantic Segmentation | 2 | 3.39% |
| Image Classification | 2 | 3.39% |
| Graph Representation Learning | 1 | 1.69% |
| Click-Through Rate Prediction | 1 | 1.69% |
Usage Over Time
Components
| Component | Type |
|
|---|---|---|
| 🤖 No Components Found | You can add them if they exist; e.g. Mask R-CNN uses RoIAlign |
