Language Models
Gated Convolution Network
Introduced by Dauphin et al. in Language Modeling with Gated Convolutional NetworksA Gated Convolutional Network is a type of language model that combines convolutional networks with a gating mechanism. Zero padding is used to ensure future context can not be seen. Gated convolutional layers can be stacked on top of other hierarchically. Model predictions are then obtained with an adaptive softmax layer.
Source: Language Modeling with Gated Convolutional Networks
Papers
| Paper | Code | Results | Date | Stars |
|---|
Tasks
| Task | Papers | Share |
|---|---|---|
| Continual Learning | 1 | 25.00% |
| Reinforcement Learning (RL) | 1 | 25.00% |
| Language Modelling | 1 | 25.00% |
| Sentence | 1 | 25.00% |
Adaptive Softmax
Gated Convolution
Gradient Clipping
Kaiming Initialization
Linear Layer
Nesterov Accelerated Gradient
Residual Connection
