Regularization
Dropout
Introduced by Srivastava et al. in Dropout: A Simple Way to Prevent Neural Networks from OverfittingDropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is $p=0.5$). At test time, all units are present, but with weights scaled by $p$ (i.e. $w$ becomes $pw$).
The idea is to prevent co-adaptation, where the neural network becomes too reliant on particular connections, as this could be symptomatic of overfitting. Intuitively, dropout can be thought of as creating an implicit ensemble of neural networks.
Source: Dropout: A Simple Way to Prevent Neural Networks from Overfitting
Papers
| Paper | Code | Results | Date | Stars |
|---|
Tasks
| Task | Papers | Share |
|---|---|---|
| Language Modelling | 49 | 6.59% |
| Retrieval | 31 | 4.17% |
| Large Language Model | 28 | 3.76% |
| Question Answering | 28 | 3.76% |
| Decoder | 21 | 2.82% |
| In-Context Learning | 13 | 1.75% |
| Text Generation | 13 | 1.75% |
| Semantic Segmentation | 13 | 1.75% |
| Image Classification | 12 | 1.61% |
Usage Over Time
Components
| Component | Type |
|
|---|---|---|
| 🤖 No Components Found | You can add them if they exist; e.g. Mask R-CNN uses RoIAlign |
