Search Results for author:
Found 6 papers, 3 papers with code
Unlocking the Potential of Multimodal Unified Discrete Representation through Training-Free Codebook Optimization and Hierarchical Alignment
The Dual Cross-modal Information Disentanglement (DCID) model, utilizing a unified codebook, shows promising results in achieving fine-grained representation and cross-modal generalization.
Language-Codec: Reducing the Gaps Between Discrete Codec Representation and Speech Language Models
Furthermore, we also validate the efficiency of the Language-Codec on downstream speech language models.
MobileSpeech: A Fast and High-Fidelity Framework for Mobile Zero-Shot Text-to-Speech
Moreover, to bridge the gap between text and speech, we introduce a high-level probabilistic mask that simulates the progression of information flow from less to more during speech generation.
TextrolSpeech: A Text Style Control Speech Corpus With Codec Language Text-to-Speech Models
The dataset comprises 236, 220 pairs of style prompt in natural text descriptions with five style factors and corresponding speech samples.
Mega-TTS 2: Boosting Prompting Mechanisms for Zero-Shot Speech Synthesis
However, the prompting mechanisms of zero-shot TTS still face challenges in the following aspects: 1) previous works of zero-shot TTS are typically trained with single-sentence prompts, which significantly restricts their performance when the data is relatively sufficient during the inference stage.
Mega-TTS: Zero-Shot Text-to-Speech at Scale with Intrinsic Inductive Bias
3) We further use a VQGAN-based acoustic model to generate the spectrogram and a latent code language model to fit the distribution of prosody, since prosody changes quickly over time in a sentence, and language models can capture both local and long-range dependencies.
