Multi-Stage Multi-Modal Pre-Training for Automatic Speech Recognition

no code implementations • 28 Mar 2024 • Yash Jain, David Chan, Pranav Dheram, Aparna Khare, Olabanji Shonibare, Venkatesh Ravichandran, Shalini Ghosh

Recent advances in machine learning have demonstrated that multi-modal pre-training can improve automatic speech recognition (ASR) performance compared to randomly initialized models, even when models are fine-tuned on uni-modal tasks.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +2

Turn-taking and Backchannel Prediction with Acoustic and Large Language Model Fusion

no code implementations • 26 Jan 2024 • Jinhan Wang, Long Chen, Aparna Khare, Anirudh Raju, Pranav Dheram, Di He, Minhua Wu, Andreas Stolcke, Venkatesh Ravichandran

We propose an approach for continuous prediction of turn-taking and backchanneling locations in spoken dialogue by fusing a neural acoustic model with a large language model (LLM).

Language Modelling Large Language Model

Mining Duplicate Questions of Stack Overflow

no code implementations • 4 Oct 2022 • Mihir Kale, Anirudha Rayasam, Radhika Parik, Pranav Dheram

There has a been a significant rise in the use of Community Question Answering sites (CQAs) over the last decade owing primarily to their ability to leverage the wisdom of the crowd.

Community Question Answering

Toward Fairness in Speech Recognition: Discovery and mitigation of performance disparities

no code implementations • 22 Jul 2022 • Pranav Dheram, Murugesan Ramakrishnan, Anirudh Raju, I-Fan Chen, Brian King, Katherine Powell, Melissa Saboowala, Karan Shetty, Andreas Stolcke

As for other forms of AI, speech recognition has recently been examined with respect to performance disparities across different user cohorts.

Fairness speech-recognition +1

On joint training with interfaces for spoken language understanding

no code implementations • 30 Jun 2021 • Anirudh Raju, Milind Rao, Gautam Tiwari, Pranav Dheram, Bryan Anderson, Zhe Zhang, Chul Lee, Bach Bui, Ariya Rastrow

Spoken language understanding (SLU) systems extract both text transcripts and semantics associated with intents and slots from input speech utterances.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +3

Do as I mean, not as I say: Sequence Loss Training for Spoken Language Understanding

no code implementations • 12 Feb 2021 • Milind Rao, Pranav Dheram, Gautam Tiwari, Anirudh Raju, Jasha Droppo, Ariya Rastrow, Andreas Stolcke

Spoken language understanding (SLU) systems extract transcriptions, as well as semantics of intent or named entities from speech, and are essential components of voice activated systems.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +3

Speech To Semantics: Improve ASR and NLU Jointly via All-Neural Interfaces

no code implementations • 14 Aug 2020 • Milind Rao, Anirudh Raju, Pranav Dheram, Bach Bui, Ariya Rastrow

Finally, we contrast these methods to a jointly trained end-to-end joint SLU model, consisting of ASR and NLU subsystems which are connected by a neural network based interface instead of text, that produces transcripts as well as NLU interpretation.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +4