Search Results for author: David M. Blei

Found 104 papers, 49 papers with code

Batch and match: black-box variational inference with a score-based divergence

no code implementations22 Feb 2024 Diana Cai, Chirag Modi, Loucas Pillaud-Vivien, Charles C. Margossian, Robert M. Gower, David M. Blei, Lawrence K. Saul

We analyze the convergence of BaM when the target distribution is Gaussian, and we prove that in the limit of infinite batch size the variational parameter updates converge exponentially quickly to the target mean and covariance.

Variational Inference

Hierarchical Causal Models

1 code implementation10 Jan 2024 Eli N. Weinstein, David M. Blei

Scientists often want to learn about cause and effect from hierarchical data, collected from subunits nested inside units.

Causal Identification

Revisiting Topic-Guided Language Models

1 code implementation4 Dec 2023 Carolina Zheng, Keyon Vafa, David M. Blei

A recent line of work in natural language processing has aimed to combine language models and topic models.

Language Modelling Topic Models

A Computational Approach to Style in American Poetry

no code implementations13 Oct 2023 David M. Kaplan, David M. Blei

We develop a quantitative method to assess the style of American poems and to visualize a collection of poems in relation to one another.

Evaluating the Moral Beliefs Encoded in LLMs

1 code implementation NeurIPS 2023 Nino Scherrer, Claudia Shi, Amir Feder, David M. Blei

(2) We apply this method to study what moral beliefs are encoded in different LLMs, especially in ambiguous cases where the right choice is not obvious.

Moral Scenarios

Amortized Variational Inference: When and Why?

2 code implementations20 Jul 2023 Charles C. Margossian, David M. Blei

We then show, on a broader class of models, how to expand the domain of AVI's inference function to improve its solution, and we provide examples, e. g. hidden Markov models, where the amortization gap cannot be closed.

Bayesian Inference Gaussian Processes +1

Density Uncertainty Layers for Reliable Uncertainty Estimation

1 code implementation21 Jun 2023 Yookoon Park, David M. Blei

In this paper, we propose a novel criterion for reliable predictive uncertainty: a model's predictive variance should be grounded in the empirical density of the input.

Out-of-Distribution Detection

An Invariant Learning Characterization of Controlled Text Generation

1 code implementation31 May 2023 Carolina Zheng, Claudia Shi, Keyon Vafa, Amir Feder, David M. Blei

In this paper, we show that the performance of controlled generation may be poor if the distributions of text in response to user prompts differ from the distribution the predictor was trained on.

Attribute Language Modelling +2

Posterior Collapse and Latent Variable Non-identifiability

no code implementations NeurIPS 2021 Yixin Wang, David M. Blei, John P. Cunningham

Unfortunately, variational autoencoders often suffer from posterior collapse: the posterior of the latent variables is equal to its prior, rendering the variational autoencoder useless as a means to produce meaningful representations.

Attribute Variational Inference

Causal Fairness Assessment of Treatment Allocation with Electronic Health Records

no code implementations21 Nov 2022 Linying Zhang, Lauren R. Richter, Yixin Wang, Anna Ostropolets, Noemie Elhadad, David M. Blei, George Hripcsak

Healthcare continues to grapple with the persistent issue of treatment disparities, sparking concerns regarding the equitable allocation of treatments in clinical practice.

Causal Inference Decision Making +1

Probabilistic Conformal Prediction Using Conditional Random Samples

1 code implementation14 Jun 2022 Zhendong Wang, Ruijiang Gao, Mingzhang Yin, Mingyuan Zhou, David M. Blei

This paper proposes probabilistic conformal prediction (PCP), a predictive inference algorithm that estimates a target variable by a discontinuous predictive set.

Conformal Prediction Prediction Intervals

CAREER: A Foundation Model for Labor Sequence Data

1 code implementation16 Feb 2022 Keyon Vafa, Emil Palikot, Tianyu Du, Ayush Kanodia, Susan Athey, David M. Blei

We fit CAREER to a dataset of 24 million job sequences from resumes, and adjust it on small longitudinal survey datasets.

Language Modelling Transfer Learning

Transport Score Climbing: Variational Inference Using Forward KL and Adaptive Neural Transport

1 code implementation3 Feb 2022 Liyi Zhang, David M. Blei, Christian A. Naesseth

Variational inference often minimizes the "reverse" Kullbeck-Leibler (KL) KL(q||p) from the approximate distribution q to the posterior p. Recent work studies the "forward" KL KL(p||q), which unlike reverse KL does not lead to variational approximations that underestimate uncertainty.

Variational Inference

Unsupervised Representation Learning via Neural Activation Coding

1 code implementation7 Dec 2021 Yookoon Park, Sangho Lee, Gunhee Kim, David M. Blei

We argue that the deep encoder should maximize its nonlinear expressivity on the data for downstream predictors to take full advantage of its representation power.

Representation Learning Retrieval

Identifiable Deep Generative Models via Sparse Decoding

1 code implementation20 Oct 2021 Gemma E. Moran, Dhanya Sridhar, Yixin Wang, David M. Blei

The underlying model is sparse in that each observed feature (i. e. each dimension of the data) depends on a small subset of the latent factors.

Representation Learning

Rationales for Sequential Predictions

2 code implementations EMNLP 2021 Keyon Vafa, Yuntian Deng, David M. Blei, Alexander M. Rush

Compared to existing baselines, greedy rationalization is best at optimizing the combinatorial objective and provides the most faithful rationales.

Combinatorial Optimization Language Modelling +2

Text-Based Ideal Points

1 code implementation ACL 2020 Keyon Vafa, Suresh Naidu, David M. Blei

In this paper, we introduce the text-based ideal point model (TBIP), an unsupervised probabilistic topic model that analyzes texts to quantify the political positions of its authors.

Towards Clarifying the Theory of the Deconfounder

no code implementations10 Mar 2020 Yixin Wang, David M. Blei

Wang and Blei (2019) studies multiple causal inference and proposes the deconfounder algorithm.

Causal Inference

Poisson-Randomized Gamma Dynamical Systems

1 code implementation NeurIPS 2019 Aaron Schein, Scott W. Linderman, Mingyuan Zhou, David M. Blei, Hanna Wallach

This paper presents the Poisson-randomized gamma dynamical system (PRGDS), a model for sequentially observed count tensors that encodes a strong inductive bias toward sparsity and burstiness.

Inductive Bias

The Blessings of Multiple Causes: A Reply to Ogburn et al. (2019)

no code implementations15 Oct 2019 Yixin Wang, David M. Blei

Ogburn et al. (2019, arXiv:1910. 05438) discuss "The Blessings of Multiple Causes" (Wang and Blei, 2018, arXiv:1805. 06826).

Population Predictive Checks

no code implementations2 Aug 2019 Gemma E. Moran, David M. Blei, Rajesh Ranganath

However, PPCs use the data twice -- both to calculate the posterior predictive and to evaluate it -- which can lead to overconfident assessments of the quality of a model.

Topic Modeling in Embedding Spaces

11 code implementations TACL 2020 Adji B. Dieng, Francisco J. R. Ruiz, David M. Blei

To this end, we develop the Embedded Topic Model (ETM), a generative model of documents that marries traditional topic models with word embeddings.

Topic Models Variational Inference +1

A Bayesian Model of Dose-Response for Cancer Drug Studies

1 code implementation10 Jun 2019 Wesley Tansey, Christopher Tosh, David M. Blei

The goal in each paired (cell line, drug) experiment is to map out the dose-response curve of the cell line as the dose level of the drug increases.

Denoising Drug Discovery +2

Adapting Neural Networks for the Estimation of Treatment Effects

5 code implementations NeurIPS 2019 Claudia Shi, David M. Blei, Victor Veitch

We propose two adaptations based on insights from the statistical literature on the estimation of treatment effects.

Causal Inference

Multiple Causes: A Causal Graphical View

no code implementations30 May 2019 Yixin Wang, David M. Blei

Our results expand the theory in Wang & Blei (2018), justify the deconfounder for causal graphs, and extend the settings where it can be used.

Causal Inference valid

Adapting Text Embeddings for Causal Inference

4 code implementations29 May 2019 Victor Veitch, Dhanya Sridhar, David M. Blei

To address this challenge, we develop causally sufficient embeddings, low-dimensional document representations that preserve sufficient information for causal identification and allow for efficient estimation of causal effects.

Causal Identification Causal Inference +4

Variational Bayes under Model Misspecification

1 code implementation NeurIPS 2019 Yixin Wang, David M. Blei

As a consequence of these results, we find that the model misspecification error dominates the variational approximation error in VB posterior predictive distributions.

Equal Opportunity and Affirmative Action via Counterfactual Predictions

no code implementations26 May 2019 Yixin Wang, Dhanya Sridhar, David M. Blei

Machine learning (ML) can automate decision-making by learning to predict decisions from historical data.

counterfactual Decision Making +1

The Medical Deconfounder: Assessing Treatment Effects with Electronic Health Records

no code implementations3 Apr 2019 Linying Zhang, Yixin Wang, Anna Ostropolets, Jami J. Mulgrave, David M. Blei, George Hripcsak

To adjust for unobserved confounders, we develop the medical deconfounder, a machine learning algorithm that unbiasedly estimates treatment effects from EHRs.

Dose-response modeling in high-throughput cancer drug screenings: An end-to-end approach

1 code implementation13 Dec 2018 Wesley Tansey, Kathy Li, Haoran Zhang, Scott W. Linderman, Raul Rabadan, David M. Blei, Chris H. Wiggins

Personalized cancer treatments based on the molecular profile of a patient's tumor are an emerging and exciting class of treatments in oncology.

Applications

A Probabilistic Model of Cardiac Physiology and Electrocardiograms

no code implementations1 Dec 2018 Andrew C. Miller, Ziad Obermeyer, David M. Blei, John P. Cunningham, Sendhil Mullainathan

An electrocardiogram (EKG) is a common, non-invasive test that measures the electrical activity of a patient's heart.

The Holdout Randomization Test for Feature Selection in Black Box Models

3 code implementations1 Nov 2018 Wesley Tansey, Victor Veitch, Haoran Zhang, Raul Rabadan, David M. Blei

We propose the holdout randomization test (HRT), an approach to feature selection using black box predictive models.

Methodology

Learning with Reflective Likelihoods

no code implementations27 Sep 2018 Adji B. Dieng, Kyunghyun Cho, David M. Blei, Yann Lecun

Furthermore, the reflective likelihood objective prevents posterior collapse when used to train stochastic auto-encoders with amortized inference.

Attribute

The Deconfounded Recommender: A Causal Inference Approach to Recommendation

no code implementations20 Aug 2018 Yixin Wang, Dawen Liang, Laurent Charlin, David M. Blei

To this end, we develop a causal approach to recommendation, one where watching a movie is a "treatment" and a user's rating is an "outcome."

Causal Inference Recommendation Systems

Avoiding Latent Variable Collapse With Generative Skip Models

no code implementations12 Jul 2018 Adji B. Dieng, Yoon Kim, Alexander M. Rush, David M. Blei

VAEs can capture complex distributions, but they can also suffer from an issue known as "latent variable collapse," especially if the likelihood model is powerful.

Empirical Risk Minimization and Stochastic Gradient Descent for Relational Data

1 code implementation27 Jun 2018 Victor Veitch, Morgane Austern, Wenda Zhou, David M. Blei, Peter Orbanz

We solve this problem using recent ideas from graph sampling theory to (i) define an empirical risk for relational data and (ii) obtain stochastic gradients for this empirical risk that are automatically unbiased.

Graph Sampling Node Classification

Black Box FDR

no code implementations ICML 2018 Wesley Tansey, Yixin Wang, David M. Blei, Raul Rabadan

BB-FDR learns a series of black box predictive models to boost power and control the false discovery rate (FDR) at two stages of study analysis.

The Blessings of Multiple Causes

2 code implementations17 May 2018 Yixin Wang, David M. Blei

Causal inference from observational data often assumes "ignorability," that all confounders are observed.

Causal Inference

Equation Embeddings

no code implementations24 Mar 2018 Kriste Krstovski, David M. Blei

Qualitatively, we found that equation embeddings provide coherent semantic representations of equations and can capture semantic similarity to other equations and to words.

Semantic Similarity Semantic Textual Similarity

Implicit Causal Models for Genome-wide Association Studies

no code implementations ICLR 2018 Dustin Tran, David M. Blei

For the first, we describe implicit causal models, a class of causal models that leverages neural architectures with an implicit density.

Bayesian Inference

Zero-Inflated Exponential Family Embeddings

no code implementations ICML 2017 Li-Ping Liu, David M. Blei

In this paper, we develop zero-inflated embeddings, a new embedding method that is designed to learn from sparse observations.

Word Embeddings

Variational Sequential Monte Carlo

1 code implementation31 May 2017 Christian A. Naesseth, Scott W. Linderman, Rajesh Ranganath, David M. Blei

The success of variational approaches depends on (i) formulating a flexible parametric family of distributions, and (ii) optimizing the parameters to find the member of this family that most closely approximates the exact posterior.

Bayesian Inference Variational Inference

Proximity Variational Inference

1 code implementation24 May 2017 Jaan Altosaar, Rajesh Ranganath, David M. Blei

Consequently, PVI is less sensitive to initialization and optimization quirks and finds better local optima.

Variational Inference

Frequentist Consistency of Variational Bayes

no code implementations9 May 2017 Yixin Wang, David M. Blei

The theorem leverages the theoretical characterizations of frequentist variational approximations to understand asymptotic properties of VB.

Stochastic Gradient Descent as Approximate Bayesian Inference

1 code implementation13 Apr 2017 Stephan Mandt, Matthew D. Hoffman, David M. Blei

Specifically, we show how to adjust the tuning parameters of constant SGD to best match the stationary distribution to a posterior, minimizing the Kullback-Leibler divergence between these two distributions.

Bayesian Inference

Deep Probabilistic Programming

no code implementations13 Jan 2017 Dustin Tran, Matthew D. Hoffman, Rif A. Saurous, Eugene Brevdo, Kevin Murphy, David M. Blei

By treating inference as a first class citizen, on a par with modeling, we show that probabilistic programming can be as flexible and computationally efficient as traditional deep learning.

Probabilistic Programming Variational Inference

Variational Inference via $χ$-Upper Bound Minimization

no code implementations1 Nov 2016 Adji B. Dieng, Dustin Tran, Rajesh Ranganath, John Paisley, David M. Blei

In this paper we propose CHIVI, a black-box variational inference algorithm that minimizes $D_{\chi}(p || q)$, the $\chi$-divergence from $p$ to $q$.

Variational Inference

Operator Variational Inference

no code implementations NeurIPS 2016 Rajesh Ranganath, Jaan Altosaar, Dustin Tran, David M. Blei

Though this divergence has been widely used, the resultant posterior approximation can suffer from undesirable statistical properties.

Bayesian Inference Variational Inference

Recurrent switching linear dynamical systems

1 code implementation26 Oct 2016 Scott W. Linderman, Andrew C. Miller, Ryan P. Adams, David M. Blei, Liam Paninski, Matthew J. Johnson

Many natural systems, such as neurons firing in the brain or basketball teams traversing a court, give rise to time series data with complex, nonlinear dynamics.

Bayesian Inference Time Series +1

Reparameterization Gradients through Acceptance-Rejection Sampling Algorithms

2 code implementations18 Oct 2016 Christian A. Naesseth, Francisco J. R. Ruiz, Scott W. Linderman, David M. Blei

Variational inference using the reparameterization trick has enabled large-scale approximate Bayesian inference in complex probabilistic models, leveraging stochastic optimization to sidestep intractable expectations.

Bayesian Inference Stochastic Optimization +1

The Generalized Reparameterization Gradient

no code implementations NeurIPS 2016 Francisco J. R. Ruiz, Michalis K. Titsias, David M. Blei

The reparameterization gradient has become a widely used method to obtain Monte Carlo gradients to optimize the variational objective.

Variational Inference

Exponential Family Embeddings

no code implementations NeurIPS 2016 Maja R. Rudolph, Francisco J. R. Ruiz, Stephan Mandt, David M. Blei

In this paper, we develop exponential family embeddings, a class of methods that extends the idea of word embeddings to other types of high-dimensional data.

Dimensionality Reduction Movie Recommendation +3

Robust Probabilistic Modeling with Bayesian Data Reweighting

1 code implementation ICML 2017 Yixin Wang, Alp Kucukelbir, David M. Blei

We propose a way to systematically detect and mitigate mismatch of a large class of probabilistic models.

Bayesian Poisson Tucker Decomposition for Learning the Structure of International Relations

1 code implementation6 Jun 2016 Aaron Schein, Mingyuan Zhou, David M. Blei, Hanna Wallach

We introduce Bayesian Poisson Tucker decomposition (BPTD) for modeling country--country interaction event data.

Posterior Dispersion Indices

no code implementations24 May 2016 Alp Kucukelbir, David M. Blei

We propose to evaluate a model through posterior dispersion.

Overdispersed Black-Box Variational Inference

no code implementations3 Mar 2016 Francisco J. R. Ruiz, Michalis K. Titsias, David M. Blei

Instead of taking samples from the variational distribution, we use importance sampling to take samples from an overdispersed distribution in the same exponential family as the variational approximation.

Variational Inference

A Variational Analysis of Stochastic Gradient Algorithms

no code implementations8 Feb 2016 Stephan Mandt, Matthew D. Hoffman, David M. Blei

With constant learning rates, it is a stochastic process that, after an initial phase of convergence, generates samples from a stationary distribution.

Variational Inference

Variational Inference: A Review for Statisticians

6 code implementations4 Jan 2016 David M. Blei, Alp Kucukelbir, Jon D. McAuliffe

One of the core problems of modern statistics is to approximate difficult-to-compute probability densities.

Stochastic Optimization Variational Inference

The Variational Gaussian Process

no code implementations20 Nov 2015 Dustin Tran, Rajesh Ranganath, David M. Blei

Variational inference is a powerful tool for approximate inference, and it has been recently applied for representation learning with deep generative models.

Representation Learning Variational Inference

A General Method for Robust Bayesian Modeling

no code implementations17 Oct 2015 Chong Wang, David M. Blei

Robust Bayesian models are appealing alternatives to standard models, providing protection from data that contains outliers or other departures from the model assumptions.

regression Topic Models

Dynamic Poisson Factorization

no code implementations15 Sep 2015 Laurent Charlin, Rajesh Ranganath, James McInerney, David M. Blei

Models for recommender systems use latent factors to explain the preferences and behaviors of users with respect to a set of items (e. g., movies, books, academic papers).

Recommendation Systems Variational Inference

Copula variational inference

no code implementations NeurIPS 2015 Dustin Tran, David M. Blei, Edoardo M. Airoldi

We develop a general variational inference method that preserves dependency among the latent variables.

Stochastic Optimization Variational Inference

Bayesian Poisson Tensor Factorization for Inferring Multilateral Relations from Sparse Dyadic Event Counts

1 code implementation10 Jun 2015 Aaron Schein, John Paisley, David M. Blei, Hanna Wallach

We demonstrate that our model's predictive performance is better than that of standard non-negative tensor factorization methods.

Deep Exponential Families

no code implementations10 Nov 2014 Rajesh Ranganath, Linpeng Tang, Laurent Charlin, David M. Blei

We describe \textit{deep exponential families} (DEFs), a class of latent variable models that are inspired by the hidden structures used in deep neural networks.

Variational Inference

Population Empirical Bayes

1 code implementation2 Nov 2014 Alp Kucukelbir, David M. Blei

We develop population empirical Bayes (POP-EB), a hierarchical framework that explicitly models the empirical population distribution as part of Bayesian analysis.

Bayesian Inference regression +1

Structured Stochastic Variational Inference

no code implementations16 Apr 2014 Matthew D. Hoffman, David M. Blei

Stochastic variational inference makes it possible to approximate posterior distributions induced by large datasets quickly using stochastic optimization.

Stochastic Optimization Variational Inference

Black Box Variational Inference

2 code implementations31 Dec 2013 Rajesh Ranganath, Sean Gerrish, David M. Blei

We evaluate our method against the corresponding black box sampling based methods.

Stochastic Optimization Variational Inference

Scalable Recommendation with Poisson Factorization

4 code implementations7 Nov 2013 Prem Gopalan, Jake M. Hofman, David M. Blei

This is an efficient algorithm that iterates over the observed entries and adjusts an approximate posterior over the user/item representations.

Variational Inference

Efficient discovery of overlapping communities in massive networks

1 code implementation PNAS 2013 2013 Prem K. Gopalan, David M. Blei

Our approach is based on a Bayesian model of networks that allows nodes to participate in multiple communities, and a corresponding algorithm that naturally interleaves subsampling from the network and updating an estimate of its communities.

Community Detection

How They Vote: Issue-Adjusted Models of Legislative Behavior

no code implementations NeurIPS 2012 Sean Gerrish, David M. Blei

We develop a probabilistic model of legislative data that uses the text of the bills to uncover lawmakers' positions on specific political issues.

Nested Hierarchical Dirichlet Processes

no code implementations25 Oct 2012 John Paisley, Chong Wang, David M. Blei, Michael. I. Jordan

We develop a nested hierarchical Dirichlet process (nHDP) for hierarchical topic modeling.

Variational Inference

Stochastic Variational Inference

2 code implementations29 Jun 2012 Matt Hoffman, David M. Blei, Chong Wang, John Paisley

We develop stochastic variational inference, a scalable algorithm for approximating posterior distributions.

Topic Models Variational Inference

Spatial distance dependent Chinese restaurant processes for image segmentation

no code implementations NeurIPS 2011 Soumya Ghosh, Andrei B. Ungureanu, Erik B. Sudderth, David M. Blei

The distance dependent Chinese restaurant process (ddCRP) was recently introduced to accommodate random partitions of non-exchangeable data.

Image Segmentation Segmentation +2

Supervised Topic Models

1 code implementation NeurIPS 2007 David M. Blei, Jon D. McAuliffe

We introduce supervised latent Dirichlet allocation (sLDA), a statistical model of labelled documents.

regression Topic Models

Variational Inference for the Nested Chinese Restaurant Process

no code implementations NeurIPS 2009 Chong Wang, David M. Blei

The nested Chinese restaurant process (nCRP) is a powerful nonparametric Bayesian model for learning tree-based hierarchies from data.

Variational Inference

Decoupling Sparsity and Smoothness in the Discrete Hierarchical Dirichlet Process

no code implementations NeurIPS 2009 Chong Wang, David M. Blei

We present a nonparametric hierarchical Bayesian model of document collections that decouples sparsity and smoothness in the component distributions (i. e., the ``topics).

Mixed Membership Stochastic Blockmodels

no code implementations NeurIPS 2008 Edo M. Airoldi, David M. Blei, Stephen E. Fienberg, Eric P. Xing

Observations consisting of measurements on relationships for pairs of objects arise in many settings, such as protein interaction and gene regulatory networks, collections of author-recipient email, and social networks.

Variational Inference

A correlated topic model of Science

no code implementations27 Aug 2007 David M. Blei, John D. Lafferty

This limitation stems from the use of the Dirichlet distribution to model the variability among the topic proportions.

Applications

Latent Dirichlet Allocation

2 code implementations1 Jan 2003 David M. Blei, Andrew Y. Ng, Michael I. Jordan

Each topic is, in turn, modeled as an infinite mixture over an underlying set of topic probabilities.

Collaborative Filtering Text Categorization +2

Cannot find the paper you are looking for? You can Submit a new open access paper.