DistillW2V2: A Small and Streaming Wav2vec 2.0 Based ASR Model

no code implementations • 16 Mar 2023 • Yanzhe Fu, Yueteng Kang, Songjun Cao, Long Ma

In this work, we propose a two-stage knowledge distillation method to solve these two problems: the first step is to make the big and non-streaming teacher model smaller, and the second step is to make it streaming.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +2

Improving Streaming Transformer Based ASR Under a Framework of Self-supervised Learning

no code implementations • 15 Sep 2021 • Songjun Cao, Yueteng Kang, Yanzhe Fu, Xiaoshuo Xu, Sining Sun, Yike Zhang, Long Ma

Under such a framework, the neural network is usually pre-trained with massive unlabeled data and then fine-tuned with limited labeled data.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +2