Search Results for author: Amrit Nagarajan

Found 4 papers, 2 papers with code

Input Compression with Positional Consistency for Efficient Training and Inference of Transformer Neural Networks

1 code implementation22 Nov 2023 Amrit Nagarajan, Anand Raghunathan

This leads to smaller input sequences being processed by the Transformer, and hence faster training, while also alleviating overfitting by presenting each input with different compression levels.

Data Augmentation

Efficient Ensembles of Graph Neural Networks

no code implementations29 Sep 2021 Amrit Nagarajan, Jacob R. Stevens, Anand Raghunathan

In this work, we leverage the unique characteristics of GNNs to overcome these overheads, creating efficient ensemble GNNs that are faster than even single models at inference time.

Ensemble Learning Network Pruning +2

Specialized Transformers: Faster, Smaller and more Accurate NLP Models

no code implementations29 Sep 2021 Amrit Nagarajan, Sanchari Sen, Jacob R. Stevens, Anand Raghunathan

We propose a Specialization framework to create optimized transformer models for a given downstream task.

Hard Attention Quantization

AxFormer: Accuracy-driven Approximation of Transformers for Faster, Smaller and more Accurate NLP Models

1 code implementation7 Oct 2020 Amrit Nagarajan, Sanchari Sen, Jacob R. Stevens, Anand Raghunathan

We propose AxFormer, a systematic framework that applies accuracy-driven approximations to create optimized transformer models for a given downstream task.

Hard Attention Quantization +1

Cannot find the paper you are looking for? You can Submit a new open access paper.