Browse State-of-the-Art
Datasets
Methods
More
Newsletter
RC2022
About
Trends
Portals
Libraries
Sign In
Subscribe to the PwC Newsletter
×
Stay informed on the latest trending ML papers with code, research developments, libraries, methods, and datasets.
Read previous issues
Join the community
×
You need to
log in
to edit.
You can
create a new account
if you don't have one.
Edit Category
×
Description with markdown (optional):
Image
Currently:
method_collections/Screen_Shot_2020-06-01_at_10.36.26_PM_FONaGgE.png
Clear
Change:
Autoregressive Transformers
Edit
Natural Language Processing
•
Transformers
• 17 methods
Methods
Add a Method
Method
Year
Papers
Transformer
Attention Is All You Need
2017
9338
GPT-3
Language Models are Few-Shot Learners
2020
1324
GPT
Improving Language Understanding by Generative Pre-Training
2018
654
GPT-2
Language Models are Unsupervised Multitask Learners
2019
543
XLNet
XLNet: Generalized Autoregressive Pretraining for Language Understanding
2019
156
Transformer-XL
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context
2019
61
Sparse Transformer
Generating Long Sequences with Sparse Transformers
2019
35
Universal Transformer
Universal Transformers
2018
17
Linformer
Linformer: Self-Attention with Linear Complexity
2020
13
Levenshtein Transformer
Levenshtein Transformer
2019
11
Primer
Primer: Searching for Efficient Transformers for Language Modeling
2021
10
Adaptive Span Transformer
Adaptive Attention Span in Transformers
2019
3
Routing Transformer
Efficient Content-Based Sparse Attention with Routing Transformers
2020
3
Feedback Transformer
Addressing Some Limitations of Transformers with Feedback Memory
2020
2
Sinkhorn Transformer
Sparse Sinkhorn Attention
2020
1
DeLighT
DeLighT: Deep and Light-weight Transformer
2020
1
Sandwich Transformer
Improving Transformer Models by Reordering their Sublayers
2019
1