Transformers

Transformers are a type of neural network architecture that have several properties that make them effective for modeling data with long-range dependencies. They generally feature a combination of multi-headed attention mechanisms, residual connections, layer normalization, feedforward connections, and positional embeddings.

Subcategories

Method Year Papers
2017 9071
2023 5244
2018 4962
2020 1282
2019 730
2018 625
2019 541
2019 533
2019 410
2019 156
2019 152
2019 125
2020 115
2020 109
2022 105
2020 105
2020 77
2020 76
2020 72
2020 63
2019 61
2020 59
2019 55
2019 48
2020 48
2019 43
2019 34
2020 29
2019 29
2000 28
2019 21
2021 20
2018 17
2021 16
2020 15
2020 14
2020 13
2019 11
2021 9
2020 9
2020 6
2021 6
2022 6
2021 6
2020 5
2021 5
2021 5
2020 5
2021 3
2019 3
2021 3
2019 3
2021 3
2019 3
2020 3
2020 3
2020 2
2020 2
2019 2
2021 2
2020 2
2021 2
2020 2
2021 2
2022 2
2021 2
2020 1
2020 1
2021 1
2021 1
2019 1
2020 1
2020 1
2021 1
2020 1
2020 1
2020 1
2020 1
2018 1
2020 1
2020 1
2021 1
2021 1
2020 1
2019 1
2021 1
2020 1
2019 1