Lip Reading

16 papers with code • 3 benchmarks • 3 datasets

Lip Reading is a task to infer the speech content in a video by using only the visual information, especially the lip movements. It has many crucial applications in practice, such as assisting audio-based speech recognition, biometric authentication and aiding hearing-impaired people.

Source: Mutual Information Maximization for Effective Lip Reading

Greatest papers with code

Talking Face Generation by Adversarially Disentangled Audio-Visual Representation

Hangz-nju-cuhk/Talking-Face-Generation-DAVS 20 Jul 2018

Talking face generation aims to synthesize a sequence of face images that correspond to a clip of speech.

Lip Reading Talking Face Generation +1

Learning Individual Speaking Styles for Accurate Lip to Speech Synthesis

Rudrabha/Lip2Wav CVPR 2020

In this work, we explore the task of lip to speech synthesis, i. e., learning to generate natural speech given only the lip movements of a speaker.

Lip Reading Speaker-Specific Lip to Speech Synthesis +1

LRW-1000: A Naturally-Distributed Large-Scale Benchmark for Lip Reading in the Wild

Fengdalu/Lipreading-DenseNet3D 16 Oct 2018

It has shown a large variation in this benchmark in several aspects, including the number of samples in each class, video resolution, lighting conditions, and speakers' attributes such as pose, age, gender, and make-up.

Lipreading Lip Reading +1

Lipreading using Temporal Convolutional Networks

mpc001/Lipreading_using_Temporal_Convolutional_Networks 23 Jan 2020

We present results on the largest publicly-available datasets for isolated word recognition in English and Mandarin, LRW and LRW1000, respectively.

Lipreading Lip Reading

Combining Residual Networks with LSTMs for Lipreading

tstafylakis/Lipreading-ResNet 12 Mar 2017

We propose an end-to-end deep learning architecture for word-level visual speech recognition.

Lipreading Lip Reading +1

Learn an Effective Lip Reading Model without Pains

Fengdalu/learn-an-effective-lip-reading-model-without-pains 15 Nov 2020

Considering the non-negligible effects of these strategies and the existing tough status to train an effective lip reading model, we perform a comprehensive quantitative study and comparative analysis, for the first time, to show the effects of several different choices for lip reading.

 Ranked #1 on Lipreading on CAS-VSR-W1k (LRW-1000) (using extra training data)

Lipreading Lip Reading +1

Lip2AudSpec: Speech reconstruction from silent lip movements video

hassanhub/LipReading 26 Oct 2017

In this study, we propose a deep neural network for reconstructing intelligible speech from silent lip movement videos.

Lip Reading

Seeing wake words: Audio-visual Keyword Spotting

lilianemomeni/KWS-Net 2 Sep 2020

The goal of this work is to automatically determine whether and when a word of interest is spoken by a talking face, with or without the audio.

Keyword Spotting Lip Reading

XFlow: Cross-modal Deep Neural Networks for Audiovisual Classification

catalina17/XFlow 2 Sep 2017

Our work improves on existing multimodal deep learning algorithms in two essential ways: (1) it presents a novel method for performing cross-modality (before features are learned from individual modalities) and (2) extends the previously proposed cross-connections which only transfer information between streams that process compatible data.

Classification General Classification +2

Mutual Information Maximization for Effective Lip Reading

xing96/MIM-lipreading 13 Mar 2020

By combining these two advantages together, the proposed method is expected to be both discriminative and robust for effective lip reading.

Lipreading Lip Reading