Search Results for author: Kavya Srinet

Found 14 papers, 7 papers with code

AnyMAL: An Efficient and Scalable Any-Modality Augmented Language Model

no code implementations27 Sep 2023 Seungwhan Moon, Andrea Madotto, Zhaojiang Lin, Tushar Nagarajan, Matt Smith, Shashank Jain, Chun-Fu Yeh, Prakash Murugesan, Peyman Heidari, Yue Liu, Kavya Srinet, Babak Damavandi, Anuj Kumar

We present Any-Modality Augmented Language Model (AnyMAL), a unified model that reasons over diverse input modality signals (i. e. text, image, video, audio, IMU motion sensor), and generates textual responses.

Language Modelling Video Question Answering

A Data Source for Reasoning Embodied Agents

1 code implementation14 Sep 2023 Jack Lanchantin, Sainbayar Sukhbaatar, Gabriel Synnaeve, Yuxuan Sun, Kavya Srinet, Arthur Szlam

In this work, to further pursue these advances, we introduce a new data generator for machine reasoning that integrates with an embodied agent.

Many Episode Learning in a Modular Embodied Agent via End-to-End Interaction

no code implementations19 Apr 2022 Yuxuan Sun, Ethan Carlson, Rebecca Qian, Kavya Srinet, Arthur Szlam

In this work we give a case study of an embodied machine-learning (ML) powered agent that improves itself via interactions with crowd-workers.

droidlet: modular, heterogenous, multi-modal agents

1 code implementation25 Jan 2021 Anurag Pratik, Soumith Chintala, Kavya Srinet, Dhiraj Gandhi, Rebecca Qian, Yuxuan Sun, Ryan Drew, Sara Elkafrawy, Anoushka Tiwari, Tucker Hart, Mary Williamson, Abhinav Gupta, Arthur Szlam

In recent years, there have been significant advances in building end-to-end Machine Learning (ML) systems that learn at scale.

CraftAssist Instruction Parsing: Semantic Parsing for a Voxel-World Assistant

no code implementations ACL 2020 Kavya Srinet, Yacine Jernite, Jonathan Gray, Arthur Szlam

We propose a semantic parsing dataset focused on instruction-driven communication with an agent in the game Minecraft.

Semantic Parsing

Why Build an Assistant in Minecraft?

1 code implementation22 Jul 2019 Arthur Szlam, Jonathan Gray, Kavya Srinet, Yacine Jernite, Armand Joulin, Gabriel Synnaeve, Douwe Kiela, Haonan Yu, Zhuoyuan Chen, Siddharth Goyal, Demi Guo, Danielle Rothermel, C. Lawrence Zitnick, Jason Weston

In this document we describe a rationale for a research program aimed at building an open "assistant" in the game Minecraft, in order to make progress on the problems of natural language understanding and learning from dialogue.

Natural Language Understanding

CraftAssist: A Framework for Dialogue-enabled Interactive Agents

3 code implementations19 Jul 2019 Jonathan Gray, Kavya Srinet, Yacine Jernite, Haonan Yu, Zhuoyuan Chen, Demi Guo, Siddharth Goyal, C. Lawrence Zitnick, Arthur Szlam

This paper describes an implementation of a bot assistant in Minecraft, and the tools and platform allowing players to interact with the bot and to record those interactions.

CraftAssist Instruction Parsing: Semantic Parsing for a Minecraft Assistant

no code implementations17 Apr 2019 Yacine Jernite, Kavya Srinet, Jonathan Gray, Arthur Szlam

We propose a large scale semantic parsing dataset focused on instruction-driven communication with an agent in Minecraft.

Semantic Parsing

Trace norm regularization and faster inference for embedded speech recognition RNNs

no code implementations ICLR 2018 Markus Kliegl, Siddharth Goyal, Kexin Zhao, Kavya Srinet, Mohammad Shoeybi

We propose and evaluate new techniques for compressing and speeding up dense matrix multiplications as found in the fully connected and recurrent layers of neural networks for embedded large vocabulary continuous speech recognition (LVCSR).

speech-recognition Speech Recognition

Cannot find the paper you are looking for? You can Submit a new open access paper.