Search Results for author:
Found 8 papers, 3 papers with code
The Larger the Better? Improved LLM Code-Generation via Budget Reallocation
On the other hand, in scenarios where unit-tests are unavailable, a ranking-based selection of candidates from the smaller model falls short of the performance of a single output from larger ones.
Transformers are Multi-State RNNs
We further show that pretrained transformers can be converted into $\textit{finite}$ multi-state RNNs by fixing the size of their hidden state.
EXPRESSO: A Benchmark and Analysis of Discrete Expressive Speech Resynthesis
Recent work has shown that it is possible to resynthesize high-quality speech based, not on text, but on low bitrate discrete units that have been learned in a self-supervised fashion and can therefore capture expressive aspects of speech that are hard to transcribe (prosody, voice styles, non-verbal vocalization).
Finding the SWEET Spot: Analysis and Improvement of Adaptive Inference in Low Resource Settings
Adaptive inference is a simple method for reducing inference costs.
Textually Pretrained Speech Language Models
In this work, we propose TWIST, a method for training SpeechLMs using a warm-start from a pretrained textual language models.
How Much Does Attention Actually Attend? Questioning the Importance of Attention in Pretrained Transformers
Our results motivate research on simpler alternatives to input-dependent attention, as well as on methods for better utilization of this mechanism in the Transformer architecture.
Efficient Methods for Natural Language Processing: A Survey
Recent work in natural language processing (NLP) has yielded appealing results from scaling model parameters and training data; however, using only scale to improve performance means that resource consumption also grows.
More than Words: In-the-Wild Visually-Driven Prosody for Text-to-Speech
In this paper we present VDTTS, a Visually-Driven Text-to-Speech model.
