no code implementations • 13 Mar 2023 • Shrihari Sridharan, Jacob R. Stevens, Kaushik Roy, Anand Raghunathan
Transformers have achieved great success in a wide variety of natural language processing (NLP) tasks due to the attention mechanism, which assigns an importance score for every word relative to other words in a sequence.
no code implementations • 29 Sep 2021 • Amrit Nagarajan, Jacob R. Stevens, Anand Raghunathan
In this work, we leverage the unique characteristics of GNNs to overcome these overheads, creating efficient ensemble GNNs that are faster than even single models at inference time.
no code implementations • 29 Sep 2021 • Amrit Nagarajan, Sanchari Sen, Jacob R. Stevens, Anand Raghunathan
We propose a Specialization framework to create optimized transformer models for a given downstream task.
1 code implementation • 7 Oct 2020 • Amrit Nagarajan, Sanchari Sen, Jacob R. Stevens, Anand Raghunathan
We propose AxFormer, a systematic framework that applies accuracy-driven approximations to create optimized transformer models for a given downstream task.