Search Results for author:
Found 29 papers, 16 papers with code
Dual-path Mamba: Short and Long-term Bidirectional Selective Structured State Space Models for Speech Separation
In this work, we replace transformers with Mamba, a selective state space model, for speech separation.
Listen, Chat, and Edit: Text-Guided Soundscape Modification for Enhanced Auditory Experience
In daily life, we encounter a variety of sounds, both desirable and undesirable, with limited control over their presence and volume.
Contextual Feature Extraction Hierarchies Converge in Large Language Models and the Brain
We also compare the feature extraction pathways of the LLMs to each other and identify new ways in which high-performing models have converged toward similar hierarchical processing mechanisms.
Exploring Self-Supervised Contrastive Learning of Spatial Sound Event Representation
In this study, we present a simple multi-channel framework for contrastive learning (MC-SimCLR) to encode 'what' and 'where' of spatial audios.
HiFTNet: A Fast High-Quality Neural Vocoder with Harmonic-plus-Noise Filter and Inverse Short Time Fourier Transform
Subjective evaluations on LJSpeech show that our model significantly outperforms both iSTFTNet and HiFi-GAN, achieving ground-truth-level performance.
SLMGAN: Exploiting Speech Language Model Representations for Unsupervised Zero-Shot Voice Conversion in GANs
In recent years, large-scale pre-trained speech language models (SLMs) have demonstrated remarkable advancements in various generative speech modeling applications, such as text-to-speech synthesis, voice conversion, and speech enhancement.
StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Models
In this paper, we present StyleTTS 2, a text-to-speech (TTS) model that leverages style diffusion and adversarial training with large speech language models (SLMs) to achieve human-level TTS synthesis.
DeCoR: Defy Knowledge Forgetting by Predicting Earlier Audio Codes
Lifelong audio feature extraction involves learning new sound classes incrementally, which is essential for adapting to new data distributions over time.
naplib-python: Neural Acoustic Data Processing and Analysis Tools in Python
Recently, the computational neuroscience community has pushed for more transparent and reproducible methods across the field.
Online Binaural Speech Separation of Moving Speakers With a Wavesplit Network
Binaural speech separation in real-world scenarios often involves moving speakers.
Improved Decoding of Attentional Selection in Multi-Talker Environments with Self-Supervised Learned Speech Representation
Auditory attention decoding (AAD) is a technique used to identify and amplify the talker that a listener is focused on in a noisy environment.
Phoneme-Level BERT for Enhanced Prosody of Text-to-Speech with Grapheme Predictions
Large-scale pre-trained language models have been shown to be helpful in improving the naturalness of text-to-speech (TTS) models by enabling them to produce more naturalistic prosodic patterns.
StyleTTS-VC: One-Shot Voice Conversion by Knowledge Transfer from Style-Based TTS Models
Here, we propose a novel approach to learning disentangled speech representation by transfer learning from style-based text-to-speech (TTS) models.
StyleTTS: A Style-Based Generative Model for Natural and Diverse Text-to-Speech Synthesis
Text-to-Speech (TTS) has recently seen great progress in synthesizing high-quality speech owing to the rapid development of parallel TTS systems, but producing speech with naturalistic prosodic variations, speaking styles and emotional tones remains challenging.
Understanding Adaptive, Multiscale Temporal Integration In Deep Speech Recognition Systems
We show that training causes these integration windows to shrink at early layers and expand at higher layers, creating a hierarchy of integration windows across the network.
StarGANv2-VC: A Diverse, Unsupervised, Non-parallel Framework for Natural-Sounding Voice Conversion
We present an unsupervised non-parallel many-to-many voice conversion (VC) method using a generative adversarial network (GAN) called StarGAN v2.
Continuous Speech Separation Using Speaker Inventory for Long Multi-talker Recording
Leveraging additional speaker information to facilitate speech separation has received increasing attention in recent years.
Group Communication with Context Codec for Lightweight Source Separation
A context codec module, containing a context encoder and a context decoder, is designed as a learnable downsampling and upsampling module to decrease the length of a sequential feature processed by the separation module.
Separating Varying Numbers of Sources with Auxiliary Autoencoding Loss
Many recent source separation systems are designed to separate a fixed number of sources out of a mixture.
End-to-end Microphone Permutation and Number Invariant Multi-channel Speech Separation
An important problem in ad-hoc microphone speech separation is how to guarantee the robustness of a system with respect to the locations and numbers of microphones.
FaSNet: Low-latency Adaptive Beamforming for Multi-microphone Audio Processing
Beamforming has been extensively investigated for multi-channel audio processing tasks.
Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separation
The majority of the previous methods have formulated the separation problem through the time-frequency representation of the mixed signal, which has several drawbacks, including the decoupling of the phase and magnitude of the signal, the suboptimality of time-frequency representation for speech separation, and the long latency in calculating the spectrograms.
Ranked #2 on
Multi-task Audio Source Seperation
on MTASS
Multi-task Audio Source Seperation
Music Source Separation
+3
Real-time Single-channel Dereverberation and Separation with Time-domainAudio Separation Network
We investigate the recently proposed Time-domain Audio Sep-aration Network (TasNet) in the task of real-time single-channel speech dereverberation.
Ranked #28 on
Speech Separation
on WSJ0-2mix
TasNet: time-domain audio separation network for real-time, single-channel speech separation
We directly model the signal in the time-domain using an encoder-decoder framework and perform the source separation on nonnegative encoder outputs.
Ranked #30 on
Speech Separation
on WSJ0-2mix
Lip2AudSpec: Speech reconstruction from silent lip movements video
In this study, we propose a deep neural network for reconstructing intelligible speech from silent lip movement videos.
Visualizing and Understanding Multilayer Perceptron Models: A Case Study in Speech Processing
Despite the recent success of deep learning, the nature of the transformations they apply to the input features remains poorly understood.
Speaker-independent Speech Separation with Deep Attractor Network
A reference point attractor is created in the embedding space to represent each speaker which is defined as the centroid of the speaker in the embedding space.
Deep attractor network for single-microphone speaker separation
We propose a novel deep learning framework for single channel speech separation by creating attractor points in high dimensional embedding space of the acoustic signals which pull together the time-frequency bins corresponding to each source.
Deep Clustering and Conventional Networks for Music Separation: Stronger Together
Deep clustering is the first method to handle general audio separation scenarios with multiple sources of the same type and an arbitrary number of sources, performing impressively in speaker-independent speech separation tasks.
