Search Results for author:
Found 30 papers, 20 papers with code
Noise2Music: Text-conditioned Music Generation with Diffusion Models
We introduce Noise2Music, where a series of diffusion models is trained to generate high-quality 30-second music clips from text prompts.
Ranked #2 on
Text-to-Music Generation
on MusicCaps
SingSong: Generating musical accompaniments from singing
We present SingSong, a system that generates instrumental music to accompany input vocals, potentially offering musicians and non-musicians alike an intuitive new way to create music featuring their own voice.
MusicLM: Generating Music From Text
We introduce MusicLM, a model generating high-fidelity music from text descriptions such as "a calming violin melody backed by a distorted guitar riff".
Ranked #8 on
Text-to-Music Generation
on MusicCaps
The Chamber Ensemble Generator: Limitless High-Quality MIR Data via Generative Modeling
We call this system the Chamber Ensemble Generator (CEG), and use it to generate a large dataset of chorales from four different chamber ensembles (CocoChorales).
Multi-instrument Music Synthesis with Spectrogram Diffusion
An ideal music synthesizer should be both interactive and expressive, generating high-fidelity audio in realtime for arbitrary combinations of instruments and notes.
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
BIG-bench focuses on tasks that are believed to be beyond the capabilities of current language models.
HEAR: Holistic Evaluation of Audio Representations
The aim of the HEAR benchmark is to develop a general-purpose audio representation that provides a strong basis for learning in a wide variety of tasks and scenarios.
General-purpose, long-context autoregressive modeling with Perceiver AR
Real-world data is high-dimensional: a book, image, or musical performance can easily contain hundreds of thousands of elements even after compression.
Ranked #35 on
Language Modelling
on WikiText-103
MIDI-DDSP: Detailed Control of Musical Performance via Hierarchical Modeling
Musical expression requires control of both what notes are played, and how they are performed.
Expressive Communication: A Common Framework for Evaluating Developments in Generative Models and Steering Interfaces
There is an increasing interest from ML and HCI communities in empowering creators with better generative models and more intuitive interfaces with which to control them.
MT3: Multi-Task Multitrack Music Transcription
Automatic Music Transcription (AMT), inferring musical notes from raw audio, is a challenging task at the core of music understanding.
Ranked #2 on
Music Transcription
on Slakh2100
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Composing Features: Compositional Model Augmentation for Steerability of Music Transformers
To tackle the combinatorial nature of composing features, we propose a compositional approach to steering music transformers, building on lightweight fine-tuning methods such as prefix tuning and bias tuning.
Sequence-to-Sequence Piano Transcription with Transformers
Automatic Music Transcription has seen significant progress in recent years by training custom deep neural networks on large datasets.
Symbolic Music Generation with Diffusion Models
Score-based generative models and diffusion probabilistic models have been successful at generating high-quality samples in continuous domains such as images and audio.
Variable-rate discrete representation learning
Semantically meaningful information content in perceptual signals is usually unevenly distributed.
Improving Perceptual Quality of Drum Transcription with the Expanded Groove MIDI Dataset
We introduce the Expanded Groove MIDI dataset (E-GMD), an automatic drum transcription (ADT) dataset that contains 444 hours of audio from 43 drum kits, making it an order of magnitude larger than similar datasets, and the first with human-performed velocity annotations.
DDSP: Differentiable Digital Signal Processing
In this paper, we introduce the Differentiable Digital Signal Processing (DDSP) library, which enables direct integration of classic signal processing elements with deep learning methods.
Encoding Musical Style with Transformer Autoencoders
We consider the problem of learning high-level controls over the global structure of generated sequences, particularly in the context of symbolic music generation with complex language models.
Learning to Groove with Inverse Sequence Transformations
We explore models for translating abstract musical ideas (scores, rhythms) into expressive performances using Seq2Seq and recurrent Variational Information Bottleneck (VIB) models.
Latent Domain Transfer: Crossing modalities with Bridging Autoencoders
We find that a simple variational autoencoder is able to learn a shared latent space to bridge between two generative models in an unsupervised fashion, and even between different types of models (ex.
GANSynth: Adversarial Neural Audio Synthesis
Efficient audio synthesis is an inherently difficult machine learning task, as human perception is sensitive to both global structure and fine-scale waveform coherence.
Latent Translation: Crossing Modalities by Bridging Generative Models
We compare to state-of-the-art techniques and find that a straight-forward variational autoencoder is able to best bridge the two generative models through learning a shared latent space.
Enabling Factorized Piano Music Modeling and Generation with the MAESTRO Dataset
Generating musical audio directly with neural networks is notoriously difficult because it requires coherently modeling structure at many different timescales.
Learning a Latent Space of Multitrack Measures
Discovering and exploring the underlying structure of multi-instrumental music using learning-based approaches remains an open problem.
A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music
The Variational Autoencoder (VAE) has proven to be an effective model for producing semantically meaningful latent representations for natural data.
Learning via social awareness: Improving a deep generative sketching model with facial feedback
We use a Latent Constraints GAN (LC-GAN) to learn from the facial feedback of a small group of viewers, by optimizing the model to produce sketches that it predicts will lead to more positive facial expressions.
Latent Constraints: Learning to Generate Conditionally from Unconditional Generative Models
Deep generative neural networks have proven effective at both conditional and unconditional modeling of complex data distributions.
Onsets and Frames: Dual-Objective Piano Transcription
We advance the state of the art in polyphonic piano music transcription by using a deep convolutional and recurrent neural network which is trained to jointly predict onsets and frames.
Neural Audio Synthesis of Musical Notes with WaveNet Autoencoders
Generative models in vision have seen rapid progress due to algorithmic improvements and the availability of high-quality image datasets.
Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
We show that an end-to-end deep learning approach can be used to recognize either English or Mandarin Chinese speech--two vastly different languages.
