Search Results for author:
Found 72 papers, 34 papers with code
Speaker Distance Estimation in Enclosures from Single-Channel Audio
Distance estimation from audio plays a crucial role in various applications, such as acoustic scene analysis, sound source localization, and room modeling.
From Weak to Strong Sound Event Labels using Adaptive Change-Point Detection and Active Learning
In this work we propose an audio recording segmentation method based on an adaptive change point detection (A-CPD) for machine guided weak label annotation of audio recording segments.
Neural Ambisonics encoding for compact irregular microphone arrays
Ambisonics encoding of microphone array signals can enable various spatial audio applications, such as virtual reality or telepresence, but it is typically designed for uniformly-spaced spherical microphone arrays.
Attention-Driven Multichannel Speech Enhancement in Moving Sound Source Scenarios
The clean speech clips from WSJ0 are employed for simulating speech signals of moving speakers in a reverberant environment.
Representation Learning for Audio Privacy Preservation using Source Separation and Robust Adversarial Learning
Privacy preservation has long been a concern in smart acoustic monitoring systems, where speech can be passively recorded along with a target signal in the system's operating environment.
Crowdsourcing and Evaluating Text-Based Audio Retrieval Relevances
Conversely, the results suggest that using only binary relevances defined by captioning-based audio-caption pairs is sufficient for contrastive learning.
STARSS23: An Audio-Visual Dataset of Spatial Recordings of Real Scenes with Spatiotemporal Annotations of Sound Events
While direction of arrival (DOA) of sound events is generally estimated from multichannel audio data recorded in a microphone array, sound events usually derive from visually perceptible source objects, e. g., sounds of footsteps come from the feet of a walker.
Permutation Invariant Recurrent Neural Networks for Sound Source Tracking Applications
Conventional recurrent neural networks (RNNs), such as the long short-term memories (LSTMs) or the gated recurrent units (GRUs), take a vector as their input and use another vector to store their state.
Simultaneous or Sequential Training? How Speech Representations Cooperate in a Multi-Task Self-Supervised Learning System
As a result, we find that sequential training with wav2vec 2. 0 first and VGS next provides higher performance on audio-visual retrieval compared to simultaneous optimization of both learning mechanisms.
Attention-Based Methods For Audio Question Answering
On the yes/no binary classification task, our proposed model achieves an accuracy of 68. 3% compared to 62. 7% in the reference model.
Adversarial Representation Learning for Robust Privacy Preservation in Audio
In this study, we propose a novel adversarial training method for learning representations of audio recordings that effectively prevents the detection of speech activity from the latent features of the recordings.
Multi-Channel Masking with Learnable Filterbank for Sound Source Separation
Specifically, each mask is used to multiply the corresponding channel's 2D representation, and the masked output of all channels are then summed.
On Negative Sampling for Contrastive Audio-Text Retrieval
With a constant training setting on the retrieval system from [1], we study eight sampling strategies, including hard and semi-hard negative sampling.
Position tracking of a varying number of sound sources with sliding permutation invariant training
Recent data- and learning-based sound source localization (SSL) methods have shown strong performance in challenging acoustic scenarios.
Language-based Audio Retrieval Task in DCASE 2022 Challenge
Language-based audio retrieval is a task, where natural language textual captions are used as queries to retrieve audio signals from a dataset.
Domestic Activity Clustering from Audio via Depthwise Separable Convolutional Autoencoder Network
Automatic estimation of domestic activities from audio can be used to solve many problems, such as reducing the labor cost for nursing the elderly people.
Language-based Audio Retrieval Task in DCASE 2022 Challenge
Language-based audio retrieval is a task, where natural language textual captions are used as queries to retrieve audio signals from a dataset.
Zero-Shot Audio Classification using Image Embeddings
The results show that the classification performance is highly sensitive to the semantic relation between test and training classes and textual and image embeddings can reach up to the semantic acoustic embeddings when the seen and unseen classes are semantically similar.
Low-complexity acoustic scene classification in DCASE 2022 Challenge
The provided baseline system is a convolutional neural network which employs post-training quantization of parameters, resulting in 46. 5 K parameters, and 29. 23 million multiply-and-accumulate operations (MMACs).
STARSS22: A dataset of spatial recordings of real scenes with spatiotemporal annotations of sound events
Additionally, the report presents the baseline system that accompanies the dataset in the challenge with emphasis on the differences with the baseline of the previous iterations; namely, introduction of the multi-ACCDOA representation to handle multiple simultaneous occurences of events of the same class, and support for additional improved input features for the microphone array format.
Ranked #1 on
Sound Event Localization and Detection
on STARSS22
Self-supervised Learning of Audio Representations from Audio-Visual Data using Spatial Alignment
In addition to the correspondence, AVSA also learns from the spatial location of acoustic and visual content.
Clotho-AQA: A Crowdsourced Dataset for Audio Question Answering
Audio question answering (AQA) is a multimodal translation task where a system analyzes an audio signal and a natural language question, to generate a desirable natural language answer.
Differentiable Tracking-Based Training of Deep Learning Sound Source Localizers
Data-based and learning-based sound source localization (SSL) has shown promising results in challenging conditions, and is commonly set as a classification or a regression problem.
Unsupervised Audio-Caption Aligning Learns Correspondences between Individual Sound Events and Textual Phrases
We investigate unsupervised learning of correspondences between sound events and textual phrases through aligning audio clips with textual captions describing the content of a whole audio clip.
Sound Event Detection: A Tutorial
The goal of automatic sound event detection (SED) methods is to recognize what is happening in an audio signal and when it is happening.
Mobile Microphone Array Speech Detection and Localization in Diverse Everyday Environments
Joint sound event localization and detection (SELD) is an integral part of developing context awareness into communication interfaces of mobile robots, smartphones, and home assistants.
Deep neural network Based Low-latency Speech Separation with Asymmetric analysis-Synthesis Window Pair
Time-frequency masking or spectrum prediction computed via short symmetric windows are commonly used in low-latency deep neural network (DNN) based source separation.
A Dataset of Dynamic Reverberant Sound Scenes with Directional Interferers for Sound Event Localization and Detection
This report presents the dataset and baseline of Task 3 of the DCASE2021 Challenge on Sound Event Localization and Detection (SELD).
Low-complexity acoustic scene classification for multi-device audio: analysis of DCASE 2021 Challenge systems
The most used techniques among the submissions were residual networks and weight quantization, with the top systems reaching over 70% accuracy, and log loss under 0. 8.
Audio-visual scene classification: analysis of DCASE 2021 Challenge submissions
More importantly, multi-modal methods using both audio and video are employed by all the top 5 teams.
Zero-Shot Audio Classification with Factored Linear and Nonlinear Acoustic-Semantic Projections
In this paper, we study zero-shot learning in audio classification through factored linear and nonlinear acoustic-semantic projections between audio instances and sound classes.
Zero-Shot Audio Classification via Semantic Embeddings
The experimental results show that classification performance is significantly improved by involving sound classes that are semantically close to the test classes in training.
Learning Contextual Tag Embeddings for Cross-Modal Alignment of Audio and Tags
In this work we propose a method for learning audio representations using an audio autoencoder (AAE), a general word embeddings model (WEM), and a multi-head self-attention (MHA) mechanism.
Neural Network-based Acoustic Vehicle Counting
This distance is predicted from audio using a two-stage (coarse-fine) regression, with both stages realised via neural networks (NNs).
Robust Audio-Based Vehicle Counting in Low-to-Moderate Traffic Flow
The method is trained and tested on a traffic-monitoring dataset comprising $422$ short, $20$-second one-channel sound files with a total of $ 1421 $ vehicles passing by the microphone.
WaveTransformer: A Novel Architecture for Audio Captioning Based on Learning Temporal and Time-Frequency Information
Automated audio captioning (AAC) is a novel task, where a method takes as an input an audio sample and outputs a textual description (i. e. a caption) of its contents.
Overview and Evaluation of Sound Event Localization and Detection in DCASE 2019
A large-scale realistic dataset of spatialized sound events was generated for the challenge, to be used for training of learning-based approaches, and for evaluation of the submissions in an unlabeled subset.
Conditioned Time-Dilated Convolutions for Sound Event Detection
Sound event detection (SED) is the task of identifying sound events along with their onset and offset times.
Multi-task Regularization Based on Infrequent Classes for Audio Captioning
Audio captioning is a multi-modal task, focusing on using natural language for describing the contents of general audio.
Temporal Sub-sampling of Audio Feature Sequences for Automated Audio Captioning
In this work we present an approach that focuses on explicitly taking advantage of this difference of lengths between sequences, by applying a temporal sub-sampling to the audio input sequence.
Depthwise Separable Convolutions Versus Recurrent Neural Networks for Monaural Singing Voice Separation
We focus on singing voice separation, employing an RNN architecture, and we replace the RNNs with DWS convolutions (DWS-CNNs).
COALA: Co-Aligned Autoencoders for Learning Semantically Enriched Audio Representations
Audio representation learning based on deep neural networks (DNNs) emerged as an alternative approach to hand-crafted features.
A Dataset of Reverberant Spatial Sound Scenes with Moving Sources for Sound Event Localization and Detection
This report presents the dataset and the evaluation setup of the Sound Event Localization & Detection (SELD) task for the DCASE 2020 Challenge.
Acoustic scene classification in DCASE 2020 Challenge: generalization across devices and low complexity solutions
This paper presents the details of Task 1: Acoustic Scene Classification in the DCASE 2020 Challenge.
Active Learning for Sound Event Detection
Training with recordings as context outperforms training with only annotated segments.
Sound Event Detection with Depthwise Separable and Dilated Convolutions
The number of the channels of the CNNs and size of the weight matrices of the RNNs have a direct effect on the total amount of parameters of the SED method, which is to a couple of millions.
Memory Requirement Reduction of Deep Neural Networks Using Low-bit Quantization of Parameters
The application of the low-bit quantization allows a 50% reduction of the DNN memory footprint while the STOI performance drops only by 2. 7%.
Clotho: An Audio Captioning Dataset
Audio captioning is the novel task of general audio content description using free text.
Crowdsourcing a Dataset of Audio Captions
In this paper we present a three steps based framework for crowdsourcing an audio captioning dataset, based on concepts and practises followed for the creation of widely used image captioning and machine translations datasets.
Sound Audio and Speech Processing
Language Modelling for Sound Event Detection with Teacher Forcing and Scheduled Sampling
On the contrary, with our method there is a decrease of 4% at F1 score and an increase of 7% at ER for the TUT-SED Synthetic 2016 dataset.
A multi-room reverberant dataset for sound event localization and detection
This paper presents the sound event localization and detection (SELD) task setup for the DCASE 2019 challenge.
Sound Audio and Speech Processing
Zero-Shot Audio Classification Based on Class Label Embeddings
We treat textual labels as semantic side information of audio classes, and use Word2Vec to generate class label embeddings.
Deep Learning for Audio Signal Processing
Given the recent surge in developments of deep learning, this article provides a review of the state-of-the-art deep learning techniques for audio signal processing.
Localization, Detection and Tracking of Multiple Moving Sound Sources with a Convolutional Recurrent Neural Network
This paper investigates the joint localization, detection, and tracking of sound events using a convolutional recurrent neural network (CRNN).
Unsupervised Adversarial Domain Adaptation Based On The Wasserstein Distance For Acoustic Scene Classification
A challenging problem in deep learning-based machine listening field is the degradation of the performance when using data from unseen conditions.
Unsupervised adversarial domain adaptation for acoustic scene classification
A general problem in acoustic scene classification task is the mismatched conditions between training and testing data, which significantly reduces the performance of the developed methods on classification accuracy.
Acoustic Scene Classification: A Competition Review
In this paper we study the problem of acoustic scene classification, i. e., categorization of audio sequences into mutually exclusive classes based on their spectral content.
A multi-device dataset for urban acoustic scene classification
This paper introduces the acoustic scene classification task of DCASE 2018 Challenge and the TUT Urban Acoustic Scenes 2018 dataset provided for the task, and evaluates the performance of a baseline system in the task.
Sound Event Localization and Detection of Overlapping Sources Using Convolutional Recurrent Neural Networks
In this paper, we propose a convolutional recurrent neural network for joint sound event localization and detection (SELD) of multiple overlapping sound events in three-dimensional (3D) space.
Sound Audio and Speech Processing
End-to-End Polyphonic Sound Event Detection Using Convolutional Recurrent Neural Networks with Learned Time-Frequency Representation Input
Sound event detection systems typically consist of two stages: extracting hand-crafted features from the raw audio waveform, and learning a mapping between these features and the target sound events using a classifier.
MaD TwinNet: Masker-Denoiser Architecture with Twin Networks for Monaural Sound Source Separation
Current state of the art (SOTA) results in monaural singing voice separation are obtained with deep learning based methods.
Sound Audio and Speech Processing
Multichannel Sound Event Detection Using 3D Convolutional Neural Networks for Learning Inter-channel Features
Each of this dataset has a four-channel first-order Ambisonic, binaural, and single-channel versions, on which the performance of SED using the proposed method are compared to study the potential of SED using multichannel audio.
Monaural Singing Voice Separation with Skip-Filtering Connections and Recurrent Inference of Time-Frequency Mask
Singing voice separation based on deep learning relies on the usage of time-frequency masking.
Sound Audio and Speech Processing
Direction of arrival estimation for multiple sound sources using convolutional recurrent neural network
This paper proposes a deep neural network for estimating the directions of arrival (DOA) of multiple sound sources.
Automated Audio Captioning with Recurrent Neural Networks
The encoder is a multi-layered, bi-directional gated recurrent unit (GRU) and the decoder a multi-layered GRU with a classification layer connected to the last GRU of the decoder.
Stacked Convolutional and Recurrent Neural Networks for Bird Audio Detection
This paper studies the detection of bird calls in audio segments using stacked convolutional and recurrent neural networks.
Sound Event Detection in Multichannel Audio Using Spatial and Harmonic Features
In this paper, we propose the use of spatial and harmonic features in combination with long short term memory (LSTM) recurrent neural network (RNN) for automatic sound event detection (SED) task.
Stacked Convolutional and Recurrent Neural Networks for Music Emotion Recognition
This paper studies the emotion recognition from musical tracks in the 2-dimensional valence-arousal (V-A) emotional space.
Sound Event Detection Using Spatial Features and Convolutional Recurrent Neural Network
This paper proposes to use low-level spatial features extracted from multichannel audio for sound event detection.
Convolutional Recurrent Neural Networks for Bird Audio Detection
Bird sounds possess distinctive spectral structure which may exhibit small shifts in spectrum depending on the bird species and environmental conditions.
Convolutional Recurrent Neural Networks for Polyphonic Sound Event Detection
Sound events often occur in unstructured environments where they exhibit wide variations in their frequency content and temporal structure.
Recurrent Neural Networks for Polyphonic Sound Event Detection in Real Life Recordings
In this paper we present an approach to polyphonic sound event detection in real life recordings based on bi-directional long short term memory (BLSTM) recurrent neural networks (RNNs).
