no code implementations • 13 Nov 2023 • Moulik Choraria, Nitesh Sekhar, Yue Wu, Xu Zhang, Prateek Singhal, Lav R. Varshney
Large-scale pretraining and instruction tuning have been successful for training general-purpose language models with broad competencies.
no code implementations • 15 Jul 2023 • Sourya Basu, Moulik Choraria, Lav R. Varshney
We find limits to the Transformer architecture for language modeling and show it has a universal prediction property in an information-theoretic sense.
1 code implementation • 28 Jan 2023 • Moulik Choraria, Ibtihal Ferwana, Ankur Mani, Lav R. Varshney
Learning models that are robust to distribution shifts is a key concern in the context of their real-life applicability.
no code implementations • ICLR 2022 • Moulik Choraria, Leello Tadesse Dadi, Grigorios Chrysos, Julien Mairal, Volkan Cevher
Inspired by such studies, we conduct a spectral analysis of the Neural Tangent Kernel (NTK) of PNNs.
no code implementations • 14 Jan 2021 • Moulik Choraria, Arpan Chattopadhyay, Urbashi Mitra, Erik Strom
Each agent node computes an estimate of the process by using its sensor observation and messages obtained from neighboring nodes, via Kalman-consensus filtering.