Search Results for author:
Found 29 papers, 14 papers with code
Relevant or Random: Can LLMs Truly Perform Analogical Reasoning?
One key finding in psychology is that compared with irrelevant past experiences, recalling relevant ones can help humans better handle new tasks.
GenTranslate: Large Language Models are Generative Multilingual Speech and Machine Translators
Leveraging the rich linguistic knowledge and strong reasoning abilities of LLMs, our new paradigm can integrate the rich information in N-best candidates to generate a higher-quality translation result.
It's Never Too Late: Fusing Acoustic Information into Large Language Models for Automatic Speech Recognition
Recent studies have successfully shown that large language models (LLMs) can be successfully used for generative error correction (GER) on top of the automatic speech recognition (ASR) output.
Audio-Visual Speech Recognition
Automatic Speech Recognition
+3
Large Language Models are Efficient Learners of Noise-Robust Speech Recognition
To this end, we propose to extract a language-space noise embedding from the N-best list to represent the noise conditions of source speech, which can promote the denoising process in GER.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+6
Multichannel AV-wav2vec2: A Framework for Learning Multichannel Multi-Modal Speech Representation
Considering that visual information helps to improve speech recognition performance in noisy scenes, in this work we propose a multichannel multi-modal speech self-supervised learning framework AV-wav2vec2, which utilizes video and multichannel audio data as inputs.
Audio-Visual Speech Recognition
Automatic Speech Recognition
+7
Generative error correction for code-switching speech recognition using large language models
In this work, we propose to leverage large language models (LLMs) and lists of hypotheses generated by an ASR to address the CS problem.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
HyPoradise: An Open Baseline for Generative Speech Recognition with Large Language Models
We make our results publicly accessible for reproducible pipelines with released pre-trained models, thus providing a new evaluation paradigm for ASR error correction with LLMs.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Rep2wav: Noise Robust text-to-speech Using self-supervised representations
Noise-robust TTS models are often trained using the enhanced speech, which thus suffer from speech distortion and background noise that affect the quality of the synthesized speech.
Noise-aware Speech Enhancement using Diffusion Probabilistic Model
Specifically, we design a noise classification (NC) model to produce acoustic embedding as a noise conditioner for guiding the reverse denoising process.
Hearing Lips in Noise: Universal Viseme-Phoneme Mapping and Transfer for Robust Audio-Visual Speech Recognition
In this work, we investigate the noise-invariant visual modality to strengthen robustness of AVSR, which can adapt to any testing noises while without dependence on noisy training data, a. k. a., unsupervised noise adaptation.
MIR-GAN: Refining Frame-Level Modality-Invariant Representations with Adversarial Network for Audio-Visual Speech Recognition
In this paper, we aim to learn the shared representations across modalities to bridge their gap.
A Neural State-Space Model Approach to Efficient Speech Separation
In this work, we introduce S4M, a new efficient speech separation framework based on neural state-space models (SSM).
Eeg2vec: Self-Supervised Electroencephalographic Representation Learning
Recently, many efforts have been made to explore how the brain processes speech using electroencephalographic (EEG) signals, where deep learning-based approaches were shown to be applicable in this field.
UniS-MMC: Multimodal Classification via Unimodality-supervised Multimodal Contrastive Learning
Multimodal learning aims to imitate human beings to acquire complementary information from multiple modalities for various downstream tasks.
Cross-Modal Global Interaction and Local Alignment for Audio-Visual Speech Recognition
However, most existing AVSR approaches simply fuse the audio and visual features by concatenation, without explicit interactions to capture the deep correlations between them, which results in sub-optimal multimodal representations for downstream speech recognition task.
Audio-Visual Speech Recognition
Automatic Speech Recognition
+3
Wav2code: Restore Clean Speech Representations via Codebook Lookup for Noise-Robust ASR
Second, during finetuning we propose a Transformer-based code predictor to accurately predict clean codes by modeling the global dependency of input noisy representations, which enables discovery and restoration of high-quality clean representations with reduced distortions.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Metric-oriented Speech Enhancement using Diffusion Probabilistic Model
Deep neural network based speech enhancement technique focuses on learning a noisy-to-clean transformation supervised by paired training data.
Unsupervised Noise adaptation using Data Simulation
Deep neural network based speech enhancement approaches aim to learn a noisy-to-clean transformation using a supervised learning paradigm.
Unifying Speech Enhancement and Separation with Gradient Modulation for End-to-End Noise-Robust Speech Separation
To alleviate this problem, we propose a novel network to unify speech enhancement and separation with gradient modulation to improve noise-robustness.
Gradient Remedy for Multi-Task Learning in End-to-End Noise-Robust Speech Recognition
In this paper, we propose a simple yet effective approach called gradient remedy (GR) to solve interference between task gradients in noise-robust speech recognition, from perspectives of both angle and magnitude.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Leveraging Modality-specific Representations for Audio-visual Speech Recognition via Reinforcement Learning
Audio-visual speech recognition (AVSR) has gained remarkable success for ameliorating the noise-robustness of speech recognition.
The Second Place Solution for The 4th Large-scale Video Object Segmentation Challenge--Track 3: Referring Video Object Segmentation
The referring video object segmentation task (RVOS) aims to segment object instances in a given video referred by a language expression in all video frames.
Self-critical Sequence Training for Automatic Speech Recognition
Although automatic speech recognition (ASR) task has gained remarkable success by sequence-to-sequence models, there are two main mismatches between its training and testing that might lead to performance degradation: 1) The typically used cross-entropy criterion aims to maximize log-likelihood of the training data, while the performance is evaluated by word error rate (WER), not log-likelihood; 2) The teacher-forcing method leads to the dependence on ground truth during training, which means that model has never been exposed to its own prediction before testing.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Interactive Audio-text Representation for Automated Audio Captioning with Contrastive Learning
Automated Audio captioning (AAC) is a cross-modal task that generates natural language to describe the content of input audio.
Noise-robust Speech Recognition with 10 Minutes Unparalleled In-domain Data
Noise-robust speech recognition systems require large amounts of training data including noisy speech data and corresponding transcripts to achieve state-of-the-art performances in face of various practical environments.
Dual-Path Style Learning for End-to-End Noise-Robust Speech Recognition
Then, we propose style learning to map the fused feature close to clean feature, in order to learn latent speech information from the latter, i. e., clean "speech style".
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Off-Policy Evaluation in Partially Observed Markov Decision Processes under Sequential Ignorability
We consider off-policy evaluation of dynamic treatment rules under sequential ignorability, given an assumption that the underlying system can be modeled as a partially observed Markov decision process (POMDP).
Interactive Feature Fusion for End-to-End Noise-Robust Speech Recognition
Speech enhancement (SE) aims to suppress the additive noise from a noisy speech signal to improve the speech's perceptual quality and intelligibility.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
The USTC-NELSLIP Systems for Simultaneous Speech Translation Task at IWSLT 2021
This paper describes USTC-NELSLIP's submissions to the IWSLT2021 Simultaneous Speech Translation task.
