Feedforward Networks

Dense Connections

Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output by a weight. This means there are $n_{\text{inputs}}*n_{\text{outputs}}$ parameters, which can lead to a lot of parameters for a sizeable network.

$$h_{l} = g\left(\textbf{W}^{T}h_{l-1}\right)$$

where $g$ is an activation function.

Image Source: Deep Learning by Goodfellow, Bengio and Courville

Papers


Paper Code Results Date Stars

Tasks


Task Papers Share
Language Modelling 60 7.72%
Retrieval 31 3.99%
Large Language Model 30 3.86%
Question Answering 29 3.73%
In-Context Learning 22 2.83%
Sentence 19 2.45%
Machine Translation 15 1.93%
Translation 15 1.93%
Object Detection 14 1.80%

Components


Component Type
🤖 No Components Found You can add them if they exist; e.g. Mask R-CNN uses RoIAlign

Categories