Search Results for author:
Found 7 papers, 4 papers with code
ReadTwice: Reading Very Large Documents with Memories
Knowledge-intensive tasks such as question answering often require assimilating information from different sections of large inputs such as books or article collections.
OmniNet: Omnidirectional Representations from Transformers
In OmniNet, instead of maintaining a strictly horizontal receptive field, each token is allowed to attend to all tokens in the entire network.
Ranked #1 on
Machine Translation
on WMT2017 Russian-English
Long Range Arena : A Benchmark for Efficient Transformers
Transformers do not scale very well to long sequence lengths largely because of quadratic self-attention complexity.
Long Range Arena: A Benchmark for Efficient Transformers
In the recent months, a wide spectrum of efficient, fast Transformers have been proposed to tackle this problem, more often than not claiming superior or comparable model quality to vanilla Transformer models.
Ranked #18 on
Long-range modeling
on LRA
(Pathfinder metric)
Big Bird: Transformers for Longer Sequences
To remedy this, we propose, BigBird, a sparse attention mechanism that reduces this quadratic dependency to linear.
Ranked #1 on
Text Classification
on Arxiv HEP-TH citation graph
Fair Hierarchical Clustering
As machine learning has become more prevalent, researchers have begun to recognize the necessity of ensuring machine learning systems are fair.
ETC: Encoding Long and Structured Inputs in Transformers
Transformer models have advanced the state of the art in many Natural Language Processing (NLP) tasks.
Ranked #3 on
Question Answering
on ConditionalQA
