Search Results for author:
Found 51 papers, 20 papers with code
Diffusion Models for Audio Restoration
Here, we aim to show that diffusion models can combine the best of both worlds and offer the opportunity to design audio restoration algorithms with a good degree of interpretability and a remarkable performance in terms of sound quality.
An Analysis of the Variance of Diffusion-based Speech Enhancement
The speech enhancement performance varies depending on the choice of the stochastic differential equation that controls the evolution of the mean and the variance along the diffusion processes when adding environmental and Gaussian noise.
Single and Few-step Diffusion for Generative Speech Enhancement
While the performance of usual generative diffusion algorithms drops dramatically when lowering the number of function evaluations (NFEs) to obtain single-step diffusion, we show that our proposed method keeps a steady performance and therefore largely outperforms the diffusion baseline in this setting and also generalizes better than its predictive counterpart.
Distilling HuBERT with LSTMs via Decoupled Knowledge Distillation
Much research effort is being applied to the task of compressing the knowledge of self-supervised models, which are powerful, yet large and memory consuming.
Live Iterative Ptychography with projection-based algorithms
In this work, we demonstrate that the ptychographic phase problem can be solved in a live fashion during scanning, while data is still being collected.
EMOCONV-DIFF: Diffusion-based Speech Emotion Conversion for Non-parallel and In-the-wild Data
Subsequently, at inference, a target emotion embedding is employed to convert the emotion of the input utterance to the given target emotion.
A Flexible Online Framework for Projection-Based STFT Phase Retrieval
Several recent contributions in the field of iterative STFT phase retrieval have demonstrated that the performance of the classical Griffin-Lim method can be considerably improved upon.
Wind Noise Reduction with a Diffusion-based Stochastic Regeneration Model
We show that our stochastic regeneration model outperforms other neural-network-based wind noise reduction methods as well as purely predictive and generative models, on a dataset using simulated and real-recorded wind noise.
Diffusion Posterior Sampling for Informed Single-Channel Dereverberation
We present in this paper an informed single-channel dereverberation method based on conditional generation with diffusion models.
On the Behavior of Intrusive and Non-intrusive Speech Enhancement Metrics in Predictive and Generative Settings
Since its inception, the field of deep speech enhancement has been dominated by predictive (discriminative) approaches, such as spectral mapping or masking.
In-the-wild Speech Emotion Conversion Using Disentangled Self-Supervised Representations and Neural Vocoder-based Resynthesis
In this work, we specifically focus on in-the-wild emotion conversion where parallel data does not exist, and the problem of disentangling lexical, speaker, and emotion information arises.
Audio-Visual Speech Enhancement with Score-Based Generative Models
This paper introduces an audio-visual speech enhancement system that leverages score-based generative models, also known as diffusion models, conditioned on visual information.
Audio-Visual Speech Separation in Noisy Environments with a Lightweight Iterative Model
We propose Audio-Visual Lightweight ITerative model (AVLIT), an effective and lightweight neural network that uses Progressive Learning (PL) to perform audio-visual speech separation in noisy environments.
Leveraging Semantic Information for Efficient Self-Supervised Emotion Recognition with Audio-Textual Distilled Models
In large part due to their implicit semantic modeling, self-supervised learning (SSL) methods have significantly increased the performance of valence recognition in speech emotion recognition (SER) systems.
Integrating Uncertainty into Neural Network-based Speech Enhancement
In this paper, we study the benefits of modeling uncertainty in clean speech estimation.
Multi-channel Speech Separation Using Spatially Selective Deep Non-linear Filters
In a multi-channel separation task with multiple speakers, we aim to recover all individual speech signals from the mixture.
Partially Adaptive Multichannel Joint Reduction of Ego-noise and Environmental Noise
In this paper, we propose a multichannel partially adaptive scheme to jointly model ego-noise and environmental noise utilizing the VAE-NMF framework, where we take advantage of spatially and spectrally structured characteristics of ego-noise by pre-training the ego-noise model, while retaining the ability to adapt to unknown environmental noise.
Speech Signal Improvement Using Causal Generative Diffusion Models
In this paper, we present a causal speech signal improvement system that is designed to handle different types of distortions.
Extending DNN-based Multiplicative Masking to Deep Subband Filtering for Improved Dereverberation
We demonstrate that the resulting deep subband filtering scheme outperforms multiplicative masking for dereverberation, while leaving the denoising performance virtually the same.
Reducing the Prior Mismatch of Stochastic Differential Equations for Diffusion-based Speech Enhancement
Recently, score-based generative models have been successfully employed for the task of speech enhancement.
StoRM: A Diffusion-based Stochastic Regeneration Model for Speech Enhancement and Dereverberation
As diffusion models are generative approaches they may also produce vocalizing and breathing artifacts in adverse conditions.
Uncertainty Estimation in Deep Speech Enhancement Using Complex Gaussian Mixture Models
Single-channel deep speech enhancement approaches often estimate a single multiplicative mask to extract clean speech without a measure of its accuracy.
DriftRec: Adapting diffusion models to blind JPEG restoration
In this work, we utilize the high-fidelity generation abilities of diffusion models to solve blind JPEG restoration at high compression levels.
DiffPhase: Generative Diffusion-based STFT Phase Retrieval
Diffusion probabilistic models have been recently used in a variety of tasks, including speech enhancement and synthesis.
Analysing Diffusion-based Generative Approaches versus Discriminative Approaches for Speech Restoration
In this paper, we systematically compare the performance of generative diffusion models and discriminative approaches on different speech restoration tasks.
Spatially Selective Deep Non-linear Filters for Speaker Extraction
In a scenario with multiple persons talking simultaneously, the spatial characteristics of the signals are the most distinct feature for extracting the target signal.
Speech Enhancement and Dereverberation with Diffusion-based Generative Models
This matches our forward process which moves from clean speech to noisy speech by including a drift term.
Ranked #19 on
Speech Enhancement
on VoiceBank + DEMAND
Label Uncertainty Modeling and Prediction for Speech Emotion Recognition using t-Distributions
To address this, these emotion annotations are typically collected by multiple annotators and averaged across annotators in order to obtain labels for arousal and valence.
Insights Into Deep Non-linear Filters for Improved Multi-channel Speech Enhancement
The key advantage of using multiple microphones for speech enhancement is that spatial filtering can be used to complement the tempo-spectral processing.
Efficient Transformer-based Speech Enhancement Using Long Frames and STFT Magnitudes
The SepFormer architecture shows very good results in speech separation.
On the Role of Spatial, Spectral, and Temporal Processing for DNN-based Non-linear Multi-channel Speech Enhancement
Employing deep neural networks (DNNs) to directly learn filters for multi-channel speech enhancement has potentially two key advantages over a traditional approach combining a linear spatial filter with an independent tempo-spectral post-filter: 1) non-linear spatial filtering allows to overcome potential restrictions originating from a linear processing model and 2) joint processing of spatial and tempo-spectral information allows to exploit interdependencies between different sources of information.
Beyond Griffin-Lim: Improved Iterative Phase Retrieval for Speech
Phase retrieval is a problem encountered not only in speech and audio processing, but in many other fields such as optics.
Neural Network-augmented Kalman Filtering for Robust Online Speech Dereverberation in Noisy Reverberant Environments
In this paper, a neural network-augmented algorithm for noise-robust online dereverberation with a Kalman filtering variant of the weighted prediction error (WPE) method is proposed.
A neural network-supported two-stage algorithm for lightweight dereverberation on hearing devices
By deriving new metrics analyzing the dereverberation performance in various time ranges, we confirm that directly optimizing for a criterion at the output of the multi-channel linear filtering stage results in a more efficient dereverberation as compared to placing the criterion at the output of the DNN to optimize the PSD estimation.
Customizable End-to-end Optimization of Online Neural Network-supported Dereverberation for Hearing Devices
This work focuses on online dereverberation for hearing devices using the weighted prediction error (WPE) algorithm.
Speech Enhancement with Score-Based Generative Models in the Complex STFT Domain
Score-based generative models (SGMs) have recently shown impressive results for difficult generative tasks such as the unconditional and conditional generation of natural images and audio signals.
Phase-Aware Deep Speech Enhancement: It's All About The Frame Length
Algorithmic latency in speech processing is dominated by the frame length used for Fourier analysis, which in turn limits the achievable performance of magnitude-centric approaches.
Integrating Statistical Uncertainty into Neural Network-Based Speech Enhancement
Speech enhancement in the time-frequency domain is often performed by estimating a multiplicative mask to extract clean speech.
Deep Iterative Phase Retrieval for Ptychography
One of the most prominent challenges in the field of diffractive imaging is the phase retrieval (PR) problem: In order to reconstruct an object from its diffraction pattern, the inverse Fourier transform must be computed.
End-To-End Label Uncertainty Modeling for Speech-based Arousal Recognition Using Bayesian Neural Networks
At training, the network learns a distribution of weights to capture the inherent uncertainty related to subjective arousal annotations.
Disentanglement Learning for Variational Autoencoders Applied to Audio-Visual Speech Enhancement
In this work, we propose to use an adversarial training scheme for variational autoencoders to disentangle the label from the other latent variables.
Nonlinear Spatial Filtering in Multichannel Speech Enhancement
Rather, the MMSE optimal filter is a joint spatial and spectral nonlinear function.
Variational Autoencoder for Speech Enhancement with a Noise-Aware Encoder
Recently, a generative variational autoencoder (VAE) has been proposed for speech enhancement to model speech statistics.
Guided Variational Autoencoder for Speech Enhancement With a Supervised Classifier
In this paper, we propose to guide the variational autoencoder with a supervised classifier separately trained on noisy speech.
Reinforcement Learning with Time-dependent Goals for Robotic Musicians
Reinforcement learning is a promising method to accomplish robotic control tasks.
Efficient Joint Estimation of Tracer Distribution and Background Signals in Magnetic Particle Imaging using a Dictionary Approach
Background signals are a primary source of artifacts in magnetic particle imaging and limit the sensitivity of the method since background signals are often not precisely known and vary over time.
SNR-Based Features and Diverse Training Data for Robust DNN-Based Speech Enhancement
In this paper, we address the generalization of deep neural network (DNN) based speech enhancement to unseen noise conditions for the case that training data is limited in size and diversity.
Robust Robotic Pouring using Audition and Haptics
Both network training results and robot experiments demonstrate that MP-Net is robust against noise and changes to the task and environment.
A Multi-Phase Gammatone Filterbank for Speech Separation via TasNet
In this work, we investigate if the learned encoder of the end-to-end convolutional time domain audio separation network (Conv-TasNet) is the key to its recent success, or if the encoder can just as well be replaced by a deterministic hand-crafted filterbank.
Making Sense of Audio Vibration for Liquid Height Estimation in Robotic Pouring
PouringNet is trained on our collected real-world pouring dataset with multimodal sensing data, which contains more than 3000 recordings of audio, force feedback, video and trajectory data of the human hand that performs the pouring task.
Robotics Sound Audio and Speech Processing
DNN and CNN with Weighted and Multi-task Loss Functions for Audio Event Detection
Our proposed systems significantly outperform the challenge baseline, improving F-score from 72. 7% to 90. 0% and reducing detection error rate from 0. 53 to 0. 18 on average on the development data.
