1 code implementation • 22 Nov 2023 • Amrit Nagarajan, Anand Raghunathan
This leads to smaller input sequences being processed by the Transformer, and hence faster training, while also alleviating overfitting by presenting each input with different compression levels.
no code implementations • 29 Sep 2021 • Amrit Nagarajan, Jacob R. Stevens, Anand Raghunathan
In this work, we leverage the unique characteristics of GNNs to overcome these overheads, creating efficient ensemble GNNs that are faster than even single models at inference time.
no code implementations • 29 Sep 2021 • Amrit Nagarajan, Sanchari Sen, Jacob R. Stevens, Anand Raghunathan
We propose a Specialization framework to create optimized transformer models for a given downstream task.
1 code implementation • 7 Oct 2020 • Amrit Nagarajan, Sanchari Sen, Jacob R. Stevens, Anand Raghunathan
We propose AxFormer, a systematic framework that applies accuracy-driven approximations to create optimized transformer models for a given downstream task.