Search Results for author:
Found 34 papers, 5 papers with code
Audiobox: Unified Audio Generation with Natural Language Prompts
Research communities have made great progress over the past year advancing the performance of large scale audio generative models for a single modality (speech, sound, or music) through adopting more powerful generative models and scaling data.
Ranked #1 on
Audio Generation
on AudioCaps
Generative Pre-training for Speech with Flow Matching
Generative models have gained more and more attention in recent years for their remarkable success in tasks that required estimating and sampling data distribution to generate high-fidelity synthetic data.
Dynamic ASR Pathways: An Adaptive Masking Approach Towards Efficient Pruning of A Multilingual ASR Model
In this work, we propose the use of an adaptive masking approach in two scenarios for pruning a multilingual ASR model efficiently, each resulting in sparse monolingual models or a sparse multilingual model (named as Dynamic ASR Pathways).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Scaling Speech Technology to 1,000+ Languages
Expanding the language coverage of speech technology has the potential to improve access to information for many more people.
SpeeChain: A Speech Toolkit for Large-Scale Machine Speech Chain
This paper introduces SpeeChain, an open-source Pytorch-based toolkit designed to develop the machine speech chain for large-scale use.
Voice-preserving Zero-shot Multiple Accent Conversion
Most people who have tried to learn a foreign language would have experienced difficulties understanding or speaking with a native speaker's accent.
Massively Multilingual ASR on 70 Languages: Tokenization, Architecture, and Generalization Capabilities
Later, we use our optimal tokenization strategy to train multiple embedding and output model to further improve our result.
Learning ASR pathways: A sparse multilingual ASR model
Neural network pruning compresses automatic speech recognition (ASR) models effectively.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Nix-TTS: Lightweight and End-to-End Text-to-Speech via Module-wise Distillation
We present Nix-TTS, a lightweight TTS achieved via knowledge distillation to a high-quality yet large-sized, non-autoregressive, and end-to-end (vocoder-free) TTS teacher model.
XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale
On the CoVoST-2 speech translation benchmark, we improve the previous state of the art by an average of 7. 4 BLEU over 21 translation directions into English.
Ranked #1 on
Language Identification
on VoxLingua107
(using extra training data)
Conformer-Based Self-Supervised Learning for Non-Speech Audio Tasks
While self-supervised speech representation learning has been popular in the speech research community, very few works have comprehensively analyzed audio representation learning for non-speech audio tasks.
Ranked #5 on
Audio Classification
on Balanced Audio Set
Accent-Robust Automatic Speech Recognition Using Supervised and Unsupervised Wav2vec Embeddings
Domain-adversarial training (DAT) and multi-task learning (MTL) are two common approaches for building accent-robust ASR models.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Improved Language Identification Through Cross-Lingual Self-Supervised Learning
Language identification greatly impacts the success of downstream tasks such as automatic speech recognition.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Incremental Machine Speech Chain Towards Enabling Listening while Speaking in Real-time
By contrast, humans can listen to what hey speak in real-time, and if there is a delay in hearing, they won't be able to continue speaking.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Augmenting Images for ASR and TTS through Single-loop and Dual-loop Multimodal Chain Framework
Previous research has proposed a machine speech chain to enable automatic speech recognition (ASR) and text-to-speech synthesis (TTS) to assist each other in semi-supervised learning and to avoid the need for a large amount of paired speech and text data.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+6
Cross-Lingual Machine Speech Chain for Javanese, Sundanese, Balinese, and Bataks Speech Recognition and Synthesis
We then develop ASR and TTS of ethnic languages by utilizing Indonesian ASR and TTS in a cross-lingual machine speech chain framework with only text or only speech data removing the need for paired speech-text data of those ethnic languages.
Sequence-to-Sequence Learning via Attention Transfer for Incremental Speech Recognition
One main reason is because the model needs to decide the incremental steps and learn the transcription that aligns with the current short speech segment.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Unsupervised Learning of Disentangled Speech Content and Style Representation
We present an approach for unsupervised learning of speech representation disentangling contents and styles.
Transformer VQ-VAE for Unsupervised Unit Discovery and Speech Synthesis: ZeroSpeech 2020 Challenge
In this paper, we report our submitted system for the ZeroSpeech 2020 challenge on Track 2019.
Deja-vu: Double Feature Presentation and Iterated Loss in Deep Transformer Networks
As our motivation is to allow acoustic models to re-examine their input features in light of partial hypotheses we introduce intermediate model heads and loss function.
Transformer-based Acoustic Modeling for Hybrid Speech Recognition
We propose and evaluate transformer-based acoustic models (AMs) for hybrid speech recognition.
Ranked #23 on
Speech Recognition
on LibriSpeech test-other
(using extra training data)
Speech-to-speech Translation between Untranscribed Unknown Languages
Second, we train a sequence-to-sequence model that directly maps the source language speech to the target language's discrete representation.
Listening while Speaking and Visualizing: Improving ASR through Multimodal Chain
Previously, a machine speech chain, which is based on sequence-to-sequence deep learning, was proposed to mimic speech perception and production behavior.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+6
VQVAE Unsupervised Unit Discovery and Multi-scale Code2Spec Inverter for Zerospeech Challenge 2019
Our proposed approach significantly improved the intelligibility (in CER), the MOS, and discrimination ABX scores compared to the official ZeroSpeech 2019 baseline or even the topline.
End-to-End Feedback Loss in Speech Chain Framework via Straight-Through Estimator
In our previous work, we applied a speech chain mechanism as a semi-supervised learning.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Multi-scale Alignment and Contextual History for Attention Mechanism in Sequence-to-sequence Model
In this paper, we propose two ideas to improve sequence-to-sequence model performance by enhancing the attention module.
Machine Speech Chain with One-shot Speaker Adaptation
In the speech chain loop mechanism, ASR also benefits from the ability to further learn an arbitrary speaker's characteristics from the generated speech waveform, resulting in a significant improvement in the recognition rate.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Tensor Decomposition for Compressing Recurrent Neural Network
In the machine learning fields, Recurrent Neural Network (RNN) has become a popular architecture for sequential data modeling.
Sequence-to-Sequence ASR Optimization via Reinforcement Learning
Despite the success of sequence-to-sequence approaches in automatic speech recognition (ASR) systems, the models still suffer from several problems, mainly due to the mismatch between the training and inference conditions.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Attention-based Wav2Text with Feature Transfer Learning
In this paper, we construct the first end-to-end attention-based encoder-decoder model to process directly from raw speech waveform to the text transcription.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Listening while Speaking: Speech Chain by Deep Learning
In this paper, we take a step further and develop a closed-loop speech chain model based on deep learning.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Gated Recurrent Neural Tensor Network
Our proposed RNNs, which are called a Long-Short Term Memory Recurrent Neural Tensor Network (LSTMRNTN) and Gated Recurrent Unit Recurrent Neural Tensor Network (GRURNTN), are made by combining the LSTM and GRU RNN models with the tensor product.
Compressing Recurrent Neural Network with Tensor Train
Recurrent Neural Network (RNN) are a popular choice for modeling temporal and sequential tasks and achieve many state-of-the-art performance on various complex problems.
Local Monotonic Attention Mechanism for End-to-End Speech and Language Processing
In this paper, we propose a novel attention mechanism that has local and monotonic properties.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
