Look Once to Hear: Target Speech Hearing with Noisy Examples

1 code implementation • 10 May 2024 • Bandhav Veluri, Malek Itani, Tuochao Chen, Takuya Yoshioka, Shyamnath Gollakota

We present the first enrollment interface where the wearer looks at the target speaker for a few seconds to capture a single, short, highly noisy, binaural example of the target speaker.

Semantic Hearing: Programming Acoustic Scenes with Binaural Hearables

1 code implementation • 1 Nov 2023 • Bandhav Veluri, Malek Itani, Justin Chan, Takuya Yoshioka, Shyamnath Gollakota

To achieve this, we make two technical contributions: 1) we present the first neural network that can achieve binaural target sound extraction in the presence of interfering sounds and background noise, and 2) we design a training methodology that allows our system to generalize to real-world use.

Target Sound Extraction

Real-Time Target Sound Extraction

1 code implementation • 4 Nov 2022 • Bandhav Veluri, Justin Chan, Malek Itani, Tuochao Chen, Takuya Yoshioka, Shyamnath Gollakota

We present the first neural network model to achieve real-time and streaming target sound extraction.

Decoder Streaming Target Sound Extraction