vq-wav2vec: Self-Supervised Learning of Discrete Speech Representations
We propose vq-wav2vec to learn discrete representations of audio segments through a wav2vec-style self-supervised context prediction task. The algorithm uses either a gumbel softmax or online k-means clustering to quantize the dense representations. Discretization enables the direct application of algorithms from the NLP community which require discrete inputs. Experiments show that BERT pre-training achieves a new state of the art on TIMIT phoneme classification and WSJ speech recognition.
PDF Abstract ICLR 2020 PDF ICLR 2020 AbstractCode
Datasets
LibriSpeech
TIMIT
Results from the Paper
Ranked #2 on
Speech Recognition
on TIMIT
(using extra training data)
| Task | Dataset | Model | Metric Name | Metric Value | Global Rank | Uses Extra Training Data |
Benchmark |
|---|---|---|---|---|---|---|---|
| Speech Recognition | TIMIT | vq-wav2vec | Percentage error | 11.6 | # 2 |
