Stochastic Optimization
Stochastic Weight Averaging
Introduced by Izmailov et al. in Averaging Weights Leads to Wider Optima and Better GeneralizationStochastic Weight Averaging is an optimization procedure that averages multiple points along the trajectory of SGD, with a cyclical or constant learning rate. On the one hand it averages weights, but it also has the property that, with a cyclical or constant learning rate, SGD proposals are approximately sampling from the loss surface of the network, leading to stochastic weights and helping to discover broader optima.
Source: Averaging Weights Leads to Wider Optima and Better Generalization
Papers
| Paper | Code | Results | Date | Stars |
|---|
Tasks
| Task | Papers | Share |
|---|---|---|
| Semantic Segmentation | 5 | 10.87% |
| Image Classification | 5 | 10.87% |
| Domain Generalization | 3 | 6.52% |
| Instance Segmentation | 2 | 4.35% |
| Node Classification | 2 | 4.35% |
| Question Answering | 2 | 4.35% |
| Object Detection | 2 | 4.35% |
| Language Modelling | 1 | 2.17% |
| Clustering | 1 | 2.17% |
Usage Over Time
Components
| Component | Type |
|
|---|---|---|
| 🤖 No Components Found | You can add them if they exist; e.g. Mask R-CNN uses RoIAlign |
