Incorporating Ultrasound Tongue Images for Audio-Visual Speech Enhancement

no code implementations • 19 Sep 2023 • Rui-Chen Zheng, Yang Ai, Zhen-Hua Ling

Specifically, we guide an audio-lip speech enhancement student model to learn from a pre-trained audio-lip-tongue speech enhancement teacher model, thus transferring tongue-related knowledge.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +3

Incorporating Ultrasound Tongue Images for Audio-Visual Speech Enhancement through Knowledge Distillation

no code implementations • 24 May 2023 • Rui-Chen Zheng, Yang Ai, Zhen-Hua Ling

Audio-visual speech enhancement (AV-SE) aims to enhance degraded speech along with extra visual information such as lip videos, and has been shown to be more effective than audio-only speech enhancement.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +3

Speech Reconstruction from Silent Tongue and Lip Articulation By Pseudo Target Generation and Domain Adversarial Training

no code implementations • 12 Apr 2023 • Rui-Chen Zheng, Yang Ai, Zhen-Hua Ling

This paper studies the task of speech reconstruction from ultrasound tongue images and optical lip videos recorded in a silent speaking mode, where people only activate their intra-oral and extra-oral articulators without producing sound.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +1