Search Results for author:
Found 23 papers, 13 papers with code
Crowdsourcing and Evaluating Text-Based Audio Retrieval Relevances
Conversely, the results suggest that using only binary relevances defined by captioning-based audio-caption pairs is sufficient for contrastive learning.
Simultaneous or Sequential Training? How Speech Representations Cooperate in a Multi-Task Self-Supervised Learning System
As a result, we find that sequential training with wav2vec 2. 0 first and VGS next provides higher performance on audio-visual retrieval compared to simultaneous optimization of both learning mechanisms.
BabySLM: language-acquisition-friendly benchmark of self-supervised spoken language models
Self-supervised techniques for learning speech representations have been shown to develop linguistic competence from exposure to speech without the need for human labels.
Syllable Discovery and Cross-Lingual Generalization in a Visually Grounded, Self-Supervised Speech Model
In this paper, we show that representations capturing syllabic units emerge when training a self-supervised speech model with a visually-grounded training objective.
Evaluation of self-supervised pre-training for automatic infant movement classification using wearable movement sensors
The recently-developed infant wearable MAIJU provides a means to automatically evaluate infants' motor performance in an objective and scalable manner in out-of-hospital settings.
Analysing the Impact of Audio Quality on the Use of Naturalistic Long-Form Recordings for Infant-Directed Speech Research
Our results show that the use of modest and high audio quality naturalistic speech data result in largely similar conclusions on IDS and ADS in terms of acoustic analyses and modelling experiments.
On Negative Sampling for Contrastive Audio-Text Retrieval
With a constant training setting on the retrieval system from [1], we study eight sampling strategies, including hard and semi-hard negative sampling.
Analysis of Self-Supervised Learning and Dimensionality Reduction Methods in Clustering-Based Active Learning for Speech Emotion Recognition
In this paper, we combine CPC and multiple dimensionality reduction methods in search of functioning practices for clustering-based AL. Our experiments for simulating speech emotion recognition system deployment show that both the local and global topology of the feature space can be successfully used for AL, and that CPC can be used to improve clustering-based AL performance over traditional signal features.
Towards Learning to Speak and Hear Through Multi-Agent Communication over a Continuous Acoustic Channel
Multi-agent reinforcement learning has been used as an effective means to study emergent communication between agents, yet little focus has been given to continuous acoustic communication.
Unsupervised Audio-Caption Aligning Learns Correspondences between Individual Sound Events and Textual Phrases
We investigate unsupervised learning of correspondences between sound events and textual phrases through aligning audio clips with textual captions describing the content of a whole audio clip.
Can phones, syllables, and words emerge as side-products of cross-situational audiovisual learning? -- A computational investigation
We review the extent that the audiovisual aspect of LLH is supported by the existing computational studies.
Language-Independent Approach for Automatic Computation of Vowel Articulation Features in Dysarthric Speech Assessment
Experimental results on a Finnish PD speech corpus demonstrate the efficacy and reliability of the proposed automatic method in deriving VAI, VSA, FCR and F2i/F2u (the second formant ratio for vowels /i/ and /u/).
ZR-2021VG: Zero-Resource Speech Challenge, Visually-Grounded Language Modelling track, 2021 edition
We present the visually-grounded language modelling track that was introduced in the Zero-Resource Speech challenge, 2021 edition, 2nd round.
Evaluation of Audio-Visual Alignments in Visually Grounded Speech Models
We compare the alignment performance using our proposed evaluation metrics to the semantic retrieval task commonly used to evaluate VGS models.
Comparison of end-to-end neural network architectures and data augmentation methods for automatic infant motility assessment using wearable sensors
In addition, we explore the benefits of data augmentation methods in ideal and non-ideal recording conditions.
Automatic Analysis of the Emotional Content of Speech in Daylong Child-Centered Recordings from a Neonatal Intensive Care Unit
Researchers have recently started to study how the emotional speech heard by young infants can affect their developmental outcomes.
Zero-Shot Audio Classification with Factored Linear and Nonlinear Acoustic-Semantic Projections
In this paper, we study zero-shot learning in audio classification through factored linear and nonlinear acoustic-semantic projections between audio instances and sound classes.
Unsupervised Discovery of Recurring Speech Patterns Using Probabilistic Adaptive Metrics
One potential approach to this problem is to use dynamic time warping (DTW) to find well-aligning patterns from the speech data.
Analysis of Predictive Coding Models for Phonemic Representation Learning in Small Datasets
The present study investigates the behaviour of two predictive coding models, Autoregressive Predictive Coding and Contrastive Predictive Coding, in a phoneme discrimination task (ABX task) for two languages with different dataset sizes.
Automatic Posture and Movement Tracking of Infants with Wearable Movement Sensors
These data were manually annotated for infant posture and movement based on video recordings of the sessions, and using a novel annotation scheme specifically designed to assess the overall movement pattern of infants in the given age group.
SylNet: An Adaptable End-to-End Syllable Count Estimator for Speech
Automatic syllable count estimation (SCE) is used in a variety of applications ranging from speaking rate estimation to detecting social activity from wearable microphones or developmental research concerned with quantifying speech heard by language-learning children in different environments.
A computational model of early language acquisition from audiovisual experiences of young infants
Earlier research has suggested that human infants might use statistical dependencies between speech and non-linguistic multimodal input to bootstrap their language learning before they know how to segment words from running speech.
Blind phoneme segmentation with temporal prediction errors
Phonemic segmentation of speech is a critical step of speech recognition systems.
