Search Results for author:
Found 75 papers, 35 papers with code
Prepacking: A Simple Method for Fast Prefilling and Increased Throughput in Large Language Models
In this work, we highlight the following pitfall of prefilling: for batches containing high-varying prompt lengths, significant computation is wasted by the standard practice of padding sequences to the maximum length.
On the Challenges and Opportunities in Generative AI
The field of deep generative modeling has grown rapidly and consistently over the years.
Polynomial Semantics of Tractable Probabilistic Circuits
Probabilistic circuits compute multilinear polynomials that represent multivariate probability distributions.
A Pseudo-Semantic Loss for Autoregressive Models with Logical Constraints
Under such distributions, computing the likelihood of even simple constraints is #P-hard.
Image Inpainting via Tractable Steering of Diffusion Models
In addition to proposing a new framework for constrained image generation, this paper highlights the benefit of more tractable models and motivates the development of expressive TPMs.
A Unified Approach to Count-Based Weakly-Supervised Learning
In many cases, these weak labels dictate the frequency of each respective class over a set of instances.
Expressive Modeling Is Insufficient for Offline RL: A Tractable Inference Perspective
A popular paradigm for offline Reinforcement Learning (RL) tasks is to first fit the offline trajectories to a sequence model, and then prompt the model for actions that lead to high expected return.
High Dimensional Causal Inference with Variational Backdoor Adjustment
For example, in medical settings, backdoor adjustment can be used to control for confounding and estimate the effectiveness of a treatment.
Probabilistically Rewired Message-Passing Neural Networks
Message-passing graph neural networks (MPNNs) emerged as powerful tools for processing graph-structured input.
Scaling Integer Arithmetic in Probabilistic Programs
Distributions on integers are ubiquitous in probabilistic modeling but remain challenging for many of today's probabilistic programming languages (PPLs).
Collapsed Inference for Bayesian Deep Learning
We tackle this challenge by revealing a previously unseen connection between inference on BNNs and volume computation problems.
Tractable Control for Autoregressive Language Generation
To overcome this challenge, we propose to use tractable probabilistic models (TPMs) to impose lexical constraints in autoregressive text generation models, which we refer to as GeLaTo (Generating Language with Tractable Constraints).
Semantic Strengthening of Neuro-Symbolic Learning
Numerous neuro-symbolic approaches have recently been proposed typically with the goal of adding symbolic knowledge to the output layer of a neural network.
Mixtures of All Trees
We show that it is possible to parameterize this Mixture of All Trees (MoAT) model compactly (using a polynomial-size representation) in a way that allows for tractable likelihood computation and optimization via stochastic gradient descent.
Understanding the Distillation Process from Deep Generative Models to Tractable Probabilistic Circuits
In this paper, we theoretically and empirically discover that the performance of a PC can exceed that of its teacher model.
Certifying Fairness of Probabilistic Circuits
In this paper, we propose an algorithm to search for discrimination patterns in a general class of probabilistic models, namely probabilistic circuits.
Sparse Probabilistic Circuits via Pruning and Growing
The growing operation increases model capacity by increasing the size of the latent space.
Scaling Up Probabilistic Circuits by Latent Variable Distillation
We propose to overcome such bottleneck by latent variable distillation: we leverage the less tractable but more expressive deep generative models to provide extra supervision over the latent variables of PCs.
SIMPLE: A Gradient Estimator for $k$-Subset Sampling
$k$-subset sampling is ubiquitous in machine learning, enabling regularization and interpretability through sparsity.
Semantic Probabilistic Layers for Neuro-Symbolic Learning
We design a predictive layer for structured-output prediction (SOP) that can be plugged into any neural network guaranteeing its predictions are consistent with a set of predefined symbolic constraints.
On the Paradox of Learning to Reason from Data
Logical reasoning is needed in a wide range of NLP tasks.
Neuro-Symbolic Entropy Regularization
We propose a loss, neuro-symbolic entropy regularization, that encourages the model to confidently predict a valid object.
Lossless Compression with Probabilistic Circuits
To overcome such problems, we establish a new class of tractable lossless compression models that permit efficient encoding and decoding: Probabilistic Circuits (PCs).
Solving Marginal MAP Exactly by Probabilistic Circuit Transformations
Probabilistic circuits (PCs) are a class of tractable probabilistic models that allow efficient, often linear-time, inference of queries such as marginals and most probable explanations (MPE).
flip-hoisting: Exploiting Repeated Parameters in Discrete Probabilistic Programs
Many of today's probabilistic programming languages (PPLs) have brittle inference performance: the performance of the underlying inference algorithm is very sensitive to the precise way in which the probabilistic program is written.
Towards an Interpretable Latent Space in Structured Models for Video Prediction
We work with models which are object-centric, i. e., explicitly work with object representations, and propagate a loss in the latent space.
Tractable Regularization of Probabilistic Circuits
Instead, we re-think regularization for PCs and propose two intuitive techniques, data softening and entropy regularization, that both take advantage of PCs' tractability and still have an efficient implementation as a computation graph.
A Compositional Atlas of Tractable Circuit Operations for Probabilistic Inference
Circuit representations are becoming the lingua franca to express and reason about tractable generative and discriminative models.
Probabilistic Sufficient Explanations
Understanding the behavior of learned classifiers is an important task, and various black-box explanations, logical reasoning approaches, and model-specific methods have been proposed.
Leveraging Unlabeled Data for Entity-Relation Extraction through Probabilistic Constraint Satisfaction
We study the problem of entity-relation extraction in the presence of symbolic domain knowledge.
Tractable Computation of Expected Kernels
Computing the expectation of kernel functions is a ubiquitous task in machine learning, with applications from classical support vector machines to exploiting kernel embeddings of distributions in probabilistic modeling, statistical inference, causal discovery, and deep learning.
Probabilistic Generating Circuits
Generating functions, which are widely used in combinatorics and probability theory, encode function values into the coefficients of a polynomial.
A Compositional Atlas of Tractable Circuit Operations: From Simple Transformations to Complex Information-Theoretic Queries
Circuit representations are becoming the lingua franca to express and reason about tractable generative and discriminative models.
Probabilistic Inference with Algebraic Constraints: Theoretical Limits and Practical Approximations
Weighted model integration (WMI) is a framework to perform advanced probabilistic inference on hybrid domains, i. e., on distributions over mixed continuous-discrete random variables and in presence of complex logical and arithmetic constraints.
On the Tractability of SHAP Explanations
First, we consider fully-factorized data distributions, and show that the complexity of computing the SHAP explanation is the same as the complexity of computing the expected value of the model.
Group Fairness by Probabilistic Modeling with Latent Fair Decisions
This is often challenging as the labels in the data are biased.
Strudel: Learning Structured-Decomposable Probabilistic Circuits
Probabilistic circuits (PCs) represent a probability distribution as a computational graph.
Handling Missing Data in Decision Trees: A Probabilistic Approach
As such, handling missing data in decision trees is a well studied problem.
On the Relationship Between Probabilistic Circuits and Determinantal Point Processes
Central to this effort is the development of tractable probabilistic models (TPMs): models whose structure guarantees efficient probabilistic inference algorithms.
Counterexample-Guided Learning of Monotonic Neural Networks
Additionally, we propose a technique to use monotonicity as an inductive bias for deep learning.
On Effective Parallelization of Monte Carlo Tree Search
Second, and more importantly, we demonstrate how the proposed necessary conditions can be adopted to design more effective parallel MCTS algorithms.
Dice: Compiling Discrete Probabilistic Programs for Scalable Inference
This reduction separates the structure of the distribution from its parameters, enabling logical reasoning tools to exploit that structure for probabilistic inference.
Programming Languages
Einsum Networks: Fast and Scalable Learning of Tractable Probabilistic Circuits
Probabilistic circuits (PCs) are a promising avenue for probabilistic modeling, as they permit a wide range of exact and efficient inference routines.
Scaling up Hybrid Probabilistic Inference with Logical and Arithmetic Constraints via Message Passing
Weighted model integration (WMI) is a very appealing framework for probabilistic inference: it allows to express the complex dependencies of real-world problems where variables are both continuous and discrete, via the language of Satisfiability Modulo Theories (SMT), as well as to compute probabilistic queries with complex logical and arithmetic constraints.
Off-Policy Deep Reinforcement Learning with Analogous Disentangled Exploration
Off-policy reinforcement learning (RL) is concerned with learning a rewarding policy by executing another policy that gathers samples of experience.
Symbolic Querying of Vector Spaces: Probabilistic Databases Meets Relational Embeddings
We propose unifying techniques from probabilistic databases and relational embedding models with the goal of performing complex queries on incomplete and uncertain data.
SAM: Squeeze-and-Mimic Networks for Conditional Visual Driving Policy Learning
To learn this representation, we train a squeeze network to drive using annotations for the side task as input.
Towards Hardware-Aware Tractable Learning of Probabilistic Models
We showcase our framework on a mobile activity recognition scenario, and on a variety of benchmark datasets representative of the field of tractable learning and of the applications of interest.
On Tractable Computation of Expected Predictions
In this paper, we identify a pair of generative and discriminative models that enables tractable computation of expectations, as well as moments of any order, of the latter with respect to the former in case of regression.
Hybrid Probabilistic Inference with Logical Constraints: Tractability and Message Passing
Weighted model integration (WMI) is a very appealing framework for probabilistic inference: it allows to express the complex dependencies of real-world hybrid scenarios where variables are heterogeneous in nature (both continuous and discrete) via the language of Satisfiability Modulo Theories (SMT); as well as computing probabilistic queries with arbitrarily complex logical constraints.
Learning Fair Naive Bayes Classifiers by Discovering and Eliminating Discrimination Patterns
As machine learning is increasingly used to make real-world decisions, recent research efforts aim to define and ensure fairness in algorithmic decision making.
Smoothing Structured Decomposable Circuits
Further, for the important case of All-Marginals, we show a more efficient linear-time algorithm.
Efficient Search-Based Weighted Model Integration
Weighted model integration (WMI) extends Weighted model counting (WMC) to the integration of functions over mixed discrete-continuous domains.
Generating and Sampling Orbits for Lifted Probabilistic Inference
A key goal in the design of probabilistic inference algorithms is identifying and exploiting properties of the distribution that make inference tractable.
What to Expect of Classifiers? Reasoning about Logistic Regression with Missing Features
While discriminative classifiers often yield strong predictive performance, missing feature values at prediction time can still be a challenge.
On Constrained Open-World Probabilistic Databases
Increasing amounts of available data have led to a heightened need for representing large-scale probabilistic knowledge bases.
Scalable Rule Learning in Probabilistic Knowledge Bases
In this paper, we present SafeLearner -- a scalable solution to probabilistic KB completion that performs probabilistic rule learning using lifted probabilistic inference -- as faster approach instead of grounding.
Approximate Knowledge Compilation by Online Collapsed Importance Sampling
In particular, when the amount of exact inference is equally limited, collapsed compilation is competitive with the state of the art, and outperforms it on several benchmarks.
On Robust Trimming of Bayesian Network Classifiers
To this end, we propose a closeness metric between Bayesian classifiers, called the expected classification agreement (ECA).
A Semantic Loss Function for Deep Learning with Symbolic Knowledge
This paper develops a novel methodology for using symbolic knowledge in deep learning.
Domain Recursion for Lifted Inference with Existential Quantifiers
In this paper, we show that domain recursion can also be applied to models with existential quantifiers.
Probabilistic Program Abstractions
Abstraction is a fundamental tool for reasoning about complex systems.
Don't Fear the Bit Flips: Optimized Coding Strategies for Binary Classification
After being trained, classifiers must often operate on data that has been corrupted by noise.
New Liftable Classes for First-Order Probabilistic Inference
Statistical relational models provide compact encodings of probabilistic dependencies in relational domains, but result in highly intractable graphical models.
Tractable Learning for Complex Probability Queries
We propose a tractable learner that guarantees efficient inference for a broader class of queries.
Symmetric Weighted First-Order Model Counting
For the combined complexity, we prove that, for every fragment FO$^{k}$, $k\geq 2$, the combined complexity of FOMC (or WFOMC) is #P-complete.
Lifted Probabilistic Inference for Asymmetric Graphical Models
Lifted probabilistic inference algorithms have been successfully applied to a large number of symmetric graphical models.
Efficient Algorithms for Bayesian Network Parameter Learning from Incomplete Data
In contrast to textbook approaches such as EM and the gradient method, our approach is non-iterative, yields closed form parameter estimates, and eliminates the need for inference in a Bayesian network.
Understanding the Complexity of Lifted Inference and Asymmetric Weighted Model Counting
In this paper we study lifted inference for the Weighted First-Order Model Counting problem (WFOMC), which counts the assignments that satisfy a given sentence in first-order logic (FOL); it has applications in Statistical Relational Learning (SRL) and Probabilistic Databases (PDB).
On the Role of Canonicity in Bottom-up Knowledge Compilation
We consider the problem of bottom-up compilation of knowledge bases, which is usually predicated on the existence of a polytime function for combining compilations using Boolean operators (usually called an Apply function).
Tractability through Exchangeability: A New Perspective on Efficient Probabilistic Inference
We develop a theory of finite exchangeability and its relation to tractable probabilistic inference.
Skolemization for Weighted First-Order Model Counting
First-order model counting emerged recently as a novel reasoning task, at the core of efficient algorithms for probabilistic logics.
On the Complexity and Approximation of Binary Evidence in Lifted Inference
Recent theoretical results show, for example, that conditioning on evidence which corresponds to binary relations is #P-hard, suggesting that no lifting is to be expected in the worst case.
Inference and learning in probabilistic logic programs using weighted Boolean formulas
This paper investigates how classical inference and learning tasks known from the graphical model community can be tackled for probabilistic logic programs.
