Search Results for author:
Found 24 papers, 8 papers with code
M$^3$AV: A Multimodal, Multigenre, and Multipurpose Audio-Visual Academic Lecture Dataset
Although multiple academic video datasets have been constructed and released, few of them support both multimodal content recognition and understanding tasks, which is partially due to the lack of high-quality human annotations.
Large language models surpass human experts in predicting neuroscience results
LLMs trained on the vast scientific literature could potentially integrate noisy yet interrelated findings to forecast novel results better than human experts.
Parameter Efficient Finetuning for Speech Emotion Recognition and Domain Adaptation
Foundation models have shown superior performance for speech emotion recognition (SER).
Speech-based Slot Filling using Large Language Models
Recently, advancements in large language models (LLMs) have shown an unprecedented ability across various language tasks.
TorchAudio 2.1: Advancing speech recognition, self-supervised learning, and audio processing components for PyTorch
TorchAudio is an open-source audio and speech processing library built for PyTorch.
SALMONN: Towards Generic Hearing Abilities for Large Language Models
Hearing is arguably an essential ability of artificial intelligence (AI) agents in the physical world, which refers to the perception and understanding of general auditory information consisting of at least three types of sounds: speech, audio events, and music.
Fine-grained Audio-Visual Joint Representations for Multimodal Large Language Models
Audio-visual large language models (LLM) have drawn significant attention, yet the fine-grained combination of both input streams is rather under-explored, which is challenging but necessary for LLMs to understand general video inputs.
Conditional Diffusion Model for Target Speaker Extraction
For the reverse-time process, a parametrised score function is conditioned on a target speaker embedding to extract the target speaker from the mixture of sources.
Connecting Speech Encoder and Large Language Model for ASR
Q-Former-based LLMs can generalise well to out-of-domain datasets, where 12% relative WER reductions over the Whisper baseline ASR model were achieved on the Eval2000 test set without using any in-domain training data from Switchboard.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Affect Recognition in Conversations Using Large Language Models
This study delves into the capacity of large language models (LLMs) to recognise human affect in conversations, with a focus on both open-domain chit-chat dialogues and task-oriented dialogues.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Enhancing Quantised End-to-End ASR Models via Personalisation
Recent end-to-end automatic speech recognition (ASR) models have become increasingly larger, making them particularly challenging to be deployed on resource-constrained devices.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Cross-Utterance Conditioned VAE for Speech Generation
Experimental results on the LibriTTS datasets demonstrate that our proposed models significantly enhance speech synthesis and editing, producing more natural and expressive speech.
Knowledge-Aware Audio-Grounded Generative Slot Filling for Limited Annotated Data
In this work, we propose a Knowledge-Aware Audio-Grounded generative slot-filling framework, termed KA2G, that focuses on few-shot and zero-shot slot filling for ToD with speech input.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+6
Can Contextual Biasing Remain Effective with Whisper and GPT-2?
End-to-end automatic speech recognition (ASR) and large language models, such as Whisper and GPT-2, have recently been scaled to use vast amounts of training data.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Graph Neural Networks for Contextual ASR with the Tree-Constrained Pointer Generator
The incorporation of biasing words obtained through contextual knowledge is of paramount importance in automatic speech recognition (ASR) applications.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
End-to-end Spoken Language Understanding with Tree-constrained Pointer Generator
Specifically, a tree-constrained pointer generator (TCPGen), a powerful and efficient biasing model component, is studied, which leverages a slot shortlist with corresponding entities to extract biasing lists.
Tree-constrained Pointer Generator with Graph Neural Network Encodings for Contextual Speech Recognition
Incorporating biasing words obtained as contextual knowledge is critical for many automatic speech recognition (ASR) applications.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Minimising Biasing Word Errors for Contextual ASR with the Tree-Constrained Pointer Generator
MBWE and BLMD further improved the effectiveness of TCPGen and achieved more significant WER reductions on the biasing words.
Cross-Utterance Conditioned VAE for Non-Autoregressive Text-to-Speech
Modelling prosody variation is critical for synthesizing natural and expressive speech in end-to-end text-to-speech (TTS) systems.
Tree-constrained Pointer Generator for End-to-end Contextual Speech Recognition
Contextual knowledge is important for real-world automatic speech recognition (ASR) applications.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Combination of Deep Speaker Embeddings for Diarisation
Significant progress has recently been made in speaker diarisation after the introduction of d-vectors as speaker embeddings extracted from neural network (NN) speaker classifiers for clustering speech segments.
Generating diverse and natural text-to-speech samples using a quantized fine-grained VAE and auto-regressive prosody prior
Recent neural text-to-speech (TTS) models with fine-grained latent features enable precise control of the prosody of synthesized speech.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Fully-hierarchical fine-grained prosody modeling for interpretable speech synthesis
This paper proposes a hierarchical, fine-grained and interpretable latent variable model for prosody based on the Tacotron 2 text-to-speech model.
Speaker diarisation using 2D self-attentive combination of embeddings
This combination uses a 2-dimensional (2D) self-attentive structure, which extends the standard self-attentive layer by averaging not only across time but also across different types of embeddings.
