Search Results for author:
Found 73 papers, 34 papers with code
Generative Marginalization Models
We introduce marginalization models (MaMs), a new family of generative models for high-dimensional discrete data.
Representing and Learning Functions Invariant Under Crystallographic Groups
The linear representation generalizes the Fourier basis to crystallographically invariant basis functions.
Neuromechanical Autoencoders: Learning to Couple Elastic and Neural Network Nonlinearity
In this work, we seek to develop machine learning analogs of this process, in which we jointly learn the morphology of complex nonlinear elastic solids along with a deep neural network to control it.
Meta-PDE: Learning to Solve PDEs Quickly Without a Mesh
Partial differential equations (PDEs) are often computationally challenging to solve, and in many settings many related PDEs must be be solved either at every timestep or for a variety of candidate boundary conditions, parameters, or geometric domains.
Multi-fidelity Monte Carlo: a pseudo-marginal approach
A key challenge in applying MCMC to scientific domains is computation: the target density of interest is often a function of expensive computations, such as a high-fidelity physical simulation, an intractable integral, or a slowly-converging iterative algorithm.
ProBF: Learning Probabilistic Safety Certificates with Barrier Functions
In practice, we have inaccurate knowledge of the system dynamics, which can lead to unsafe behaviors due to unmodeled residual dynamics.
Slice Sampling Reparameterization Gradients
Slice sampling is a Markov chain Monte Carlo algorithm for simulating samples from probability distributions; it only requires a density function that can be evaluated point-wise up to a normalization constant, making it applicable to a variety of inference problems and unnormalized models.
Vitruvion: A Generative Model of Parametric CAD Sketches
Parametric computer-aided design (CAD) tools are the predominant way that engineers specify physical structures, from bicycle pedals to airplanes to printed circuit boards.
Why Generalization in RL is Difficult: Epistemic POMDPs and Implicit Partial Observability
Generalization is a central challenge for the deployment of reinforcement learning (RL) systems in the real world.
Amortized Synthesis of Constrained Configurations Using a Differentiable Surrogate
We compare our approach to direct optimization of the design using the learned surrogate, and to supervised learning of the synthesis problem.
Active multi-fidelity Bayesian online changepoint detection
We propose a multi-fidelity approach that makes cost-sensitive decisions about which data fidelity to collect based on maximizing information gain with respect to changepoints.
Task-Agnostic Amortized Inference of Gaussian Process Hyperparameters
One of the appeals of the GP framework is that the marginal likelihood of the kernel hyperparameters is often available in closed form, enabling optimization and sampling procedures to fit these hyperparameters to data.
Randomized Automatic Differentiation
The successes of deep learning, variational inference, and many other fields have been aided by specialized implementations of reverse-mode automatic differentiation (AD) to compute gradients of mega-dimensional objectives.
SketchGraphs: A Large-Scale Dataset for Modeling Relational Geometry in Computer-Aided Design
Parametric computer-aided design (CAD) is the dominant paradigm in mechanical engineering for physical design.
Learning Composable Energy Surrogates for PDE Order Reduction
We use a neural network to model the stored potential energy in a component given boundary conditions.
SUMO: Unbiased Estimation of Log Marginal Probability for Latent Variable Models
Standard variational lower bounds used to train latent variable models produce biased estimates of most quantities of interest.
On Warm-Starting Neural Network Training
We would like each of these models in the sequence to be performant and take advantage of all the data that are available to that point.
On The Difficulty of Warm-Starting Neural Network Training
We would like each of these models in the sequence to be performant and take advantage of all the data that are available to that point.
A Theoretical Connection Between Statistical Physics and Reinforcement Learning
The central object in the statistical physics abstraction is the idea of a partition function $\mathcal{Z}$, and here we construct a partition function from the ensemble of possible trajectories that an agent might take in a Markov decision process.
SpArSe: Sparse Architecture Search for CNNs on Resource-Constrained Microcontrollers
The vast majority of processors in the world are actually microcontroller units (MCUs), which find widespread use performing simple control tasks in applications ranging from automobiles to medical devices and office equipment.
Efficient Optimization of Loops and Limits with Randomized Telescoping Sums
We consider optimization problems in which the objective requires an inner loop with many steps or is the limit of a sequence of increasingly costly approximations.
A Bayesian Nonparametric View on Count-Min Sketch
The count-min sketch is a time- and memory-efficient randomized data structure that provides a point estimate of the number of times an item has appeared in a data stream.
Rapid Prediction of Electron-Ionization Mass Spectrometry using Neural Networks
When confronted with a substance of unknown identity, researchers often perform mass spectrometry on the sample and compare the observed spectrum to a library of previously-collected spectra to identify the molecule.
Motivating the Rules of the Game for Adversarial Example Research
Advances in machine learning have led to broad deployment of systems with impressive performance on important problems.
Non-Vacuous Generalization Bounds at the ImageNet Scale: A PAC-Bayesian Compression Approach
Our main technical result is a generalization bound for compressed networks based on the compressed size.
Approximate Inference for Constructing Astronomical Catalogs from Images
We present a new, fully generative model for constructing astronomical catalogs from optical telescope image sets.
Estimating the Spectral Density of Large Implicit Matrices
However, naive eigenvalue estimation is computationally expensive even when the matrix can be represented; in many of these situations the matrix is so large as to only be available implicitly via products with vectors.
PASS-GLM: polynomial approximate sufficient statistics for scalable Bayesian GLM inference
We provide theoretical guarantees on the quality of point (MAP) estimates, the approximate posterior, and posterior mean and uncertainty estimates.
Reducing Reparameterization Gradient Variance
Optimization with noisy gradients has become ubiquitous in statistics and machine learning.
Multimodal Prediction and Personalization of Photo Edits with Deep Generative Models
Professional-grade software applications are powerful but complicated$-$expert users can achieve impressive results, but novices often struggle to complete even basic tasks.
Variational Boosting: Iteratively Refining Posterior Approximations
We propose a black-box variational inference method to approximate intractable distributions with an increasingly rich approximating class.
Bayesian latent structure discovery from multi-neuron recordings
Neural circuits contain heterogeneous groups of neurons that differ in type, location, connectivity, and basic response properties.
Recurrent switching linear dynamical systems
Many natural systems, such as neurons firing in the brain or basketball teams traversing a court, give rise to time series data with complex, nonlinear dynamics.
Automatic chemical design using a data-driven continuous representation of molecules
We report a method to convert discrete representations of molecules to and from a multidimensional continuous representation.
Clustering with a Reject Option: Interactive Clustering as Bayesian Prior Elicitation
A good clustering can help a data analyst to explore and understand a data set, but what constitutes a good clustering may depend on domain-specific and application-specific criteria.
Composing graphical models with neural networks for structured representations and fast inference
We propose a general modeling and inference framework that composes probabilistic graphical models with deep learning methods and combines their respective strengths.
Patterns of Scalable Bayesian Inference
Bayesian methods are an excellent fit for this demand, but scaling Bayesian inference is a challenge.
A Gaussian Process Model of Quasar Spectral Energy Distributions
We propose a method for combining two sources of astronomical data, spectroscopy and photometry, that carry information about sources of light (e. g., stars, galaxies, and quasars) at extremely different spectral resolutions.
Dependent Multinomial Models Made Easy: Stick-Breaking with the Polya-gamma Augmentation
For example, nucleotides in a DNA sequence, children's names in a given state and year, and text documents are all commonly modeled with multinomial distributions.
A General Framework for Constrained Bayesian Optimization using Information-based Search
Of particular interest to us is to efficiently solve problems with decoupled constraints, in which subsets of the objective and constraint functions may be evaluated independently.
Predictive Entropy Search for Multi-objective Bayesian Optimization
The results show that PESMO produces better recommendations with a smaller number of evaluations of the objectives, and that a decoupled evaluation can lead to improvements in performance, particularly when the number of objectives is large.
Sandwiching the marginal likelihood using bidirectional Monte Carlo
Using the ground truth log-ML estimates obtained from our method, we quantitatively evaluate a wide variety of existing ML estimators on several latent variable models: clustering, a low rank approximation, and a binary attributes model.
Convolutional Networks on Graphs for Learning Molecular Fingerprints
We introduce a convolutional neural network that operates directly on graphs.
Ranked #2 on
Drug Discovery
on HIV dataset
Scalable Bayesian Inference for Excitatory Point Process Networks
We build on previous work that has taken a Bayesian approach to this problem, specifying prior distributions over the latent network structure and a likelihood of observed activity given this network.
Dependent Multinomial Models Made Easy: Stick Breaking with the Pólya-Gamma Augmentation
Many practical modeling problems involve discrete data that are best represented as draws from multinomial or categorical distributions.
Spectral Representations for Convolutional Neural Networks
In this work, we demonstrate that, beyond its advantages for efficient computation, the spectral domain also provides a powerful representation in which to model and train convolutional neural networks (CNNs).
Ranked #169 on
Image Classification
on CIFAR-100
Early Stopping is Nonparametric Variational Inference
By tracking the change in entropy over this sequence of transformations during optimization, we form a scalable, unbiased estimate of the variational lower bound on the log marginal likelihood.
Scalable Bayesian Optimization Using Deep Neural Networks
Bayesian optimization is an effective methodology for the global optimization of functions with expensive evaluations.
Ranked #156 on
Image Classification
on CIFAR-10
Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks
In principle, the Bayesian approach to learning neural networks does not have these problems.
Predictive Entropy Search for Bayesian Optimization with Unknown Constraints
Unknown constraints arise in many types of expensive black-box optimization problems.
Gradient-based Hyperparameter Optimization through Reversible Learning
Tuning hyperparameters of learning algorithms is hard because gradients are usually unavailable.
A framework for studying synaptic plasticity with neural spike train data
Learning and memory in the brain are implemented by complex, time-varying changes in neural circuitry.
Accelerating MCMC via Parallel Predictive Prefetching
We present a general framework for accelerating a large class of widely used Markov chain Monte Carlo (MCMC) algorithms.
Firefly Monte Carlo: Exact MCMC with Subsets of Data
Markov chain Monte Carlo (MCMC) is a popular and successful general-purpose tool for Bayesian inference.
Bayesian Optimization with Unknown Constraints
Recent work on Bayesian optimization has shown its effectiveness in global optimization of difficult black-box objective functions.
Avoiding pathologies in very deep networks
Choosing appropriate architectures and regularization strategies for deep networks is crucial to good predictive performance.
Learning the Parameters of Determinantal Point Process Kernels
Determinantal point processes (DPPs) are well-suited for modeling repulsion and have proven useful in many applications where diversity is desired.
Input Warping for Bayesian Optimization of Non-stationary Functions
Bayesian optimization has proven to be a highly effective methodology for the global optimization of unknown, expensive and multimodal functions.
Learning Ordered Representations with Nested Dropout
To learn these representations we introduce nested dropout, a procedure for stochastically removing coherent nested sets of hidden units in a neural network.
Discovering Latent Network Structure in Point Process Data
Networks play a central role in modern data analysis, enabling us to reason about systems by studying the relationships between their parts.
Message Passing Inference with Chemical Reaction Networks
We show algebraically that the steady state concentration of these species correspond to the marginal distributions of the random variables in the graph and validate the results in simulations.
A Determinantal Point Process Latent Variable Model for Inhibition in Neural Spiking Data
Point processes are popular models of neural spiking behavior as they provide a statistical distribution over temporal sequences of spikes and help to reveal the complexities underlying a series of recorded action potentials.
Multi-Task Bayesian Optimization
We demonstrate the utility of this new acquisition function by utilizing a small dataset in order to explore hyperparameter settings for a large dataset.
Ranked #93 on
Image Classification
on STL-10
Contrastive Learning Using Spectral Methods
In many natural settings, the analysis goal is not to characterize a single data set in isolation, but rather to understand the difference between one set of observations and another.
ClusterCluster: Parallel Markov Chain Monte Carlo for Dirichlet Process Mixtures
Applied to mixture modeling, our approach enables the Dirichlet process to simultaneously learn clusters that describe the data and superclusters that define the granularity of parallelization.
Priors for Diversity in Generative Latent Variable Models
We show how to perform MAP inference with DPP priors in latent Dirichlet allocation and in mixture models, leading to better intuition for the latent variable representation and quantitatively improved unsupervised feature extraction, without compromising the generative aspects of the model.
Cardinality Restricted Boltzmann Machines
The Restricted Boltzmann Machine (RBM) is a popular density model that is also good for extracting features.
Parallel MCMC with Generalized Elliptical Slice Sampling
Probabilistic models are conceptually powerful tools for finding structure in data, but their practical effectiveness is often limited by our ability to perform inference in them.
Practical Bayesian Optimization of Machine Learning Algorithms
In this work, we consider the automatic tuning problem within the framework of Bayesian optimization, in which a learning algorithm's generalization performance is modeled as a sample from a Gaussian process (GP).
Ranked #185 on
Image Classification
on CIFAR-10
Slice sampling covariance hyperparameters of latent Gaussian models
The Gaussian process (GP) is a popular way to specify dependencies between random variables in a probabilistic model.
Tree-Structured Stick Breaking for Hierarchical Data
Many data are naturally modeled by an unobserved hierarchical structure.
The Gaussian Process Density Sampler
Samples drawn from the GPDS are consistent with exact, independent samples from a fixed density function that is a transformation of a function drawn from a Gaussian process prior.
