Search Results for author:
Found 14 papers, 0 papers with code
TransformerFAM: Feedback attention is working memory
While Transformers have revolutionized deep learning, their quadratic attention complexity hinders their ability to process infinitely long inputs.
Extreme Encoder Output Frame Rate Reduction: Improving Computational Latencies of Large End-to-End Models
In the present work, we study one such strategy: applying multiple frame reduction layers in the encoder to compress encoder outputs into a small number of output frames.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Revisiting the Entropy Semiring for Neural Speech Recognition
In this work, we revisit the entropy semiring for neural speech recognition models, and show how alignment entropy can be used to supervise models through regularization or distillation.
Massive End-to-end Models for Short Search Queries
In this work, we investigate two popular end-to-end automatic speech recognition (ASR) models, namely Connectionist Temporal Classification (CTC) and RNN-Transducer (RNN-T), for offline recognition of voice search queries, with up to 2B model parameters.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Improving Speech Recognition for African American English With Audio Classification
By combining the classifier output with coarse geographic information, we can select a subset of utterances from a large corpus of untranscribed short-form queries for semi-supervised learning at scale.
Edit Distance based RL for RNNT decoding
RNN-T is currently considered the industry standard in ASR due to its exceptional WERs in various benchmark tests and its ability to support seamless streaming and longform transcription.
Modular Domain Adaptation for Conformer-Based Streaming ASR
Speech data from different domains has distinct acoustic and linguistic characteristics.
Efficient Domain Adaptation for Speech Foundation Models
The FM encoder adapter and decoder are then finetuned to the target domain with a small amount of supervised in-domain data.
Resource-Efficient Transfer Learning From Speech Foundation Model Using Hierarchical Feature Fusion
Experimental results show that the proposed method can achieve better performance on speech recognition task than existing algorithms with fewer number of trainable parameters, less computational memory cost and faster training speed.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Comparison of Soft and Hard Target RNN-T Distillation for Large-scale ASR
Knowledge distillation is an effective machine learning technique to transfer knowledge from a teacher model to a smaller student model, especially with unlabeled data.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
A Unified Cascaded Encoder ASR Model for Dynamic Model Sizes
In this paper, we propose a dynamic cascaded encoder Automatic Speech Recognition (ASR) model, which unifies models for different deployment scenarios.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Pseudo Label Is Better Than Human Label
State-of-the-art automatic speech recognition (ASR) systems are trained with tens of thousands of hours of labeled speech data.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Incremental Layer-wise Self-Supervised Learning for Efficient Speech Domain Adaptation On Device
These models are typically trained on the server using transcribed speech data.
Large-scale ASR Domain Adaptation using Self- and Semi-supervised Learning
Self- and semi-supervised learning methods have been actively investigated to reduce labeled training data or enhance the model performance.
