Search Results for author:
Found 46 papers, 13 papers with code
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
In this report, we present the latest model of the Gemini family, Gemini 1. 5 Pro, a highly compute-efficient multimodal mixture-of-experts model capable of recalling and reasoning over fine-grained information from millions of tokens of context, including multiple long documents and hours of video and audio.
Ranked #19 on
Code Generation
on HumanEval
Handling Ambiguity in Emotion: From Out-of-Domain Detection to Distribution Estimation
The evidential uncertainty measure is extended to quantify the uncertainty in emotion distribution estimation.
Gemini: A Family of Highly Capable Multimodal Models
This report introduces a new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding.
Ranked #1 on
Multi-task Language Understanding
on MMLU
(using extra training data)
SLM: Bridge the thin gap between speech and text foundation models
We present a joint Speech and Language Model (SLM), a multitask, multilingual, and dual-modal model that takes advantage of pretrained foundational speech and language models.
Efficient Adapters for Giant Speech Models
However, finetuning all parameters from the self-supervised learned model can be computationally expensive, and becomes infeasiable as the size of the model and the number of downstream tasks scales.
Google USM: Scaling Automatic Speech Recognition Beyond 100 Languages
We introduce the Universal Speech Model (USM), a single large model that performs automatic speech recognition (ASR) across 100+ languages.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Textless Direct Speech-to-Speech Translation with Discrete Speech Representation
Research on speech-to-speech translation (S2ST) has progressed rapidly in recent years.
Accented Speech Recognition: Benchmarking, Pre-training, and Diverse Data
However, there are not enough data sets for accented speech, and for the ones that are already available, more training approaches need to be explored to improve the quality of accented speech recognition.
Self-supervised Learning with Random-projection Quantizer for Speech Recognition
In particular the quantizer projects speech inputs with a randomly initialized matrix, and does a nearest-neighbor lookup in a randomly-initialized codebook.
Cross-attention conformer for context modeling in speech enhancement for ASR
This work introduces \emph{cross-attention conformer}, an attention-based architecture for context modeling in speech enhancement.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recognition
We summarize the results of a host of efforts using giant automatic speech recognition (ASR) models pre-trained using large, diverse unlabeled datasets containing approximately a million hours of audio.
Ranked #1 on
Speech Recognition
on Common Voice
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
W2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Training
In particular, when compared to published models such as conformer-based wav2vec~2. 0 and HuBERT, our model shows~5\% to~10\% relative WER reduction on the test-clean and test-other subsets.
Ranked #1 on
Speech Recognition
on LibriSpeech test-clean
(using extra training data)
Bridging the gap between streaming and non-streaming ASR systems bydistilling ensembles of CTC and RNN-T models
To improve streaming models, a recent study [1] proposed to distill a non-streaming teacher model on unsupervised utterances, and then train a streaming student using the teachers' predictions.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Pushing the Limits of Non-Autoregressive Speech Recognition
We combine recent advancements in end-to-end speech recognition to non-autoregressive automatic speech recognition.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
A Better and Faster End-to-End Model for Streaming ASR
To address this, we explore replacing the LSTM layers in the encoder of our E2E model with Conformer layers [4], which has shown good improvements for ASR.
Audio and Speech Processing Sound
Cascaded encoders for unifying streaming and non-streaming ASR
The proposed model consists of streaming and non-streaming encoders.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Improving Streaming Automatic Speech Recognition With Non-Streaming Model Distillation On Unsupervised Data
We propose a novel and effective learning method by leveraging a non-streaming ASR model as a teacher to generate transcripts on an arbitrarily large data set, which is then used to distill knowledge into streaming ASR models.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
FastEmit: Low-latency Streaming ASR with Sequence-level Emission Regularization
FastEmit also improves streaming ASR accuracy from 4. 4%/8. 9% to 3. 1%/7. 5% WER, meanwhile reduces 90th percentile latency from 210 ms to only 30 ms on LibriSpeech.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition
We employ a combination of recent developments in semi-supervised learning for automatic speech recognition to obtain state-of-the-art results on LibriSpeech utilizing the unlabeled audio of the Libri-Light dataset.
Ranked #1 on
Speech Recognition
on LibriSpeech test-clean
(using extra training data)
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Dual-mode ASR: Unify and Improve Streaming ASR with Full-context Modeling
Streaming automatic speech recognition (ASR) aims to emit each hypothesized word as quickly and accurately as possible, while full-context ASR waits for the completion of a full speech utterance before emitting completed hypotheses.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Parallel Rescoring with Transformer for Streaming On-Device Speech Recognition
The 2nd-pass model plays a key role in the quality improvement of the end-to-end model to surpass the conventional model.
Improved Noisy Student Training for Automatic Speech Recognition
Noisy student training is an iterative self-training method that leverages augmentation to improve network performance.
Ranked #5 on
Speech Recognition
on LibriSpeech test-clean
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Conformer: Convolution-augmented Transformer for Speech Recognition
Recently Transformer and Convolution neural network (CNN) based models have shown promising results in Automatic Speech Recognition (ASR), outperforming Recurrent neural networks (RNNs).
Ranked #12 on
Speech Recognition
on LibriSpeech test-other
(using extra training data)
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
ContextNet: Improving Convolutional Neural Networks for Automatic Speech Recognition with Global Context
We demonstrate that on the widely used LibriSpeech benchmark, ContextNet achieves a word error rate (WER) of 2. 1%/4. 6% without external language model (LM), 1. 9%/4. 1% with LM and 2. 9%/7. 0% with only 10M parameters on the clean/noisy LibriSpeech test sets.
Ranked #12 on
Speech Recognition
on LibriSpeech test-clean
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
RNN-T Models Fail to Generalize to Out-of-Domain Audio: Causes and Solutions
On a long-form YouTube test set, when the nonstreaming RNN-T model is trained with shorter segments of data, the proposed combination improves word error rate (WER) from 22. 3% to 14. 8%; when the streaming RNN-T model trained on short Search queries, the proposed techniques improve WER on the YouTube set from 67. 0% to 25. 3%.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
A Streaming On-Device End-to-End Model Surpassing Server-Side Conventional Model Quality and Latency
Thus far, end-to-end (E2E) models have not been shown to outperform state-of-the-art conventional models with respect to both quality, i. e., word error rate (WER), and latency, i. e., the time the hypothesis is finalized after the user stops speaking.
SpecAugment on Large Scale Datasets
Recently, SpecAugment, an augmentation scheme for automatic speech recognition that acts directly on the spectrogram of input utterances, has shown to be highly effective in enhancing the performance of end-to-end networks on public datasets.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Speech Sentiment Analysis via Pre-trained Features from End-to-end ASR Models
In this paper, we propose to use pre-trained features from end-to-end ASR models to solve speech sentiment analysis as a down-stream task.
A comparison of end-to-end models for long-form speech recognition
In this paper, we both investigate and improve the performance of end-to-end models on long-form transcription.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Recognizing long-form speech using streaming end-to-end models
In this work, we examine the ability of E2E models to generalize to unseen domains, where we find that models trained on short utterances fail to generalize to long-form speech.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Two-Pass End-to-End Speech Recognition
However, this model still lags behind a large state-of-the-art conventional model in quality [2].
Monotonic Infinite Lookback Attention for Simultaneous Machine Translation
Simultaneous machine translation begins to translate each source sentence before the source speaker is finished speaking, with applications to live and streaming scenarios.
SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition
On LibriSpeech, we achieve 6. 8% WER on test-other without the use of a language model, and 5. 8% WER with shallow fusion with a language model.
Ranked #1 on
Speech Recognition
on Hub5'00 SwitchBoard
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling
Lingvo is a Tensorflow framework offering a complete solution for collaborative deep learning research, with a particular focus towards sequence-to-sequence models.
Leveraging Weakly Supervised Data to Improve End-to-End Speech-to-Text Translation
In this paper, we demonstrate that using pre-trained MT or text-to-speech (TTS) synthesis models to convert weakly supervised data into speech-to-translation pairs for ST training can be more effective than multi-task learning.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+5
CaLcs: Continuously Approximating Longest Common Subsequence for Sequence Level Optimization
Maximum-likelihood estimation (MLE) is one of the most widely used approaches for training structured prediction models for text-generation based natural language processing applications.
A Comparison of Techniques for Language Model Integration in Encoder-Decoder Speech Recognition
In this paper, we compare a suite of past methods and some of our own proposed methods for using unpaired text data to improve encoder-decoder models.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Monotonic Chunkwise Attention
Sequence-to-sequence models with soft attention have been successfully applied to a wide variety of problems, but their decoding process incurs a quadratic time and space cost and is inapplicable to real-time sequence transduction.
Improving the Performance of Online Neural Transducer Models
Neural transducer is a streaming sequence-to-sequence model, but has shown a significant degradation in performance compared to non-streaming models such as Listen, Attend and Spell (LAS).
State-of-the-art Speech Recognition With Sequence-to-Sequence Models
Attention-based encoder-decoder architectures such as Listen, Attend, and Spell (LAS), subsume the acoustic, pronunciation and language model components of a traditional automatic speech recognition (ASR) system into a single neural network.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Minimum Word Error Rate Training for Attention-based Sequence-to-Sequence Models
Sequence-to-sequence models, such as attention-based models in automatic speech recognition (ASR), are typically trained to optimize the cross-entropy criterion which corresponds to improving the log-likelihood of the data.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
No Need for a Lexicon? Evaluating the Value of the Pronunciation Lexica in End-to-End Models
However, there has been little previous work comparing phoneme-based versus grapheme-based sub-word units in the end-to-end modeling framework, to determine whether the gains from such approaches are primarily due to the new probabilistic model, or from the joint learning of the various components with grapheme-based units.
Speech recognition for medical conversations
We explored both CTC and LAS systems for building speech recognition models.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
An online sequence-to-sequence model for noisy speech recognition
This is because the models require that the entirety of the input sequence be available at the beginning of inference, an assumption that is not valid for instantaneous speech recognition.
Learning Hard Alignments with Variational Inference
There has recently been significant interest in hard attention models for tasks such as object recognition, visual captioning and speech recognition.
Learning Online Alignments with Continuous Rewards Policy Gradient
Though capable and easy to use, they require that the entirety of the input sequence is available at the beginning of inference, an assumption that is not valid for instantaneous translation and speech recognition.
