Browse SoTA > Methodology > Mutual Information Estimation

Mutual Information Estimation

9 papers with code · Methodology

To estimate mutual information from samples, specially for high-dimensional variables.

Benchmarks

No evaluation results yet. Help compare methods by submit evaluation metrics.

Greatest papers with code

Practical and Consistent Estimation of f-Divergences

NeurIPS 2019 google-research/google-research

The estimation of an f-divergence between two probability distributions based on samples is a fundamental problem in statistics and machine learning.

MUTUAL INFORMATION ESTIMATION REPRESENTATION LEARNING

Learning deep representations by mutual information estimation and maximization

ICLR 2019 rdevon/DIM

In this work, we perform unsupervised learning of representations by maximizing mutual information between an input and the output of a deep neural network encoder.

MUTUAL INFORMATION ESTIMATION REPRESENTATION LEARNING

Empowerment-driven Exploration using Mutual Information Estimation

11 Oct 2018navneet-nmk/pytorch-rl

However, many of the state of the art deep reinforcement learning algorithms, that rely on epsilon-greedy, fail on these environments.

MONTEZUMA'S REVENGE MUTUAL INFORMATION ESTIMATION

Graph Representation Learning via Graphical Mutual Information Maximization

4 Feb 2020zpeng27/GMI

The richness in the content of various information networks such as social networks and communication networks provides the unprecedented potential for learning high-quality expressive representations without external supervision.

GRAPH REPRESENTATION LEARNING LINK PREDICTION MUTUAL INFORMATION ESTIMATION NODE CLASSIFICATION

Scalable Mutual Information Estimation using Dependence Graphs

27 Jan 2018mrtnoshad/EDGE

To the best of our knowledge EDGE is the first non-parametric MI estimator that can achieve parametric MSE rates with linear time complexity.

INFORMATION PLANE MUTUAL INFORMATION ESTIMATION

Neural Entropic Estimation: A faster path to mutual information estimation

30 May 2019ccha23/MI-NEE

In particular, we show that MI-NEE reduces to MINE in the special case when the reference distribution is the product of marginal distributions, but faster convergence is possible by choosing the uniform distribution as the reference distribution instead.

MUTUAL INFORMATION ESTIMATION

Better Long-Range Dependency By Bootstrapping A Mutual Information Regularizer

28 May 2019BorealisAI/BMI

In this work, we develop a novel regularizer to improve the learning of long-range dependency of sequence data.

LANGUAGE MODELLING MUTUAL INFORMATION ESTIMATION

Telescoping Density-Ratio Estimation

22 Jun 2020benrhodes26/tre_code

Density-ratio estimation via classification is a cornerstone of unsupervised learning.

MUTUAL INFORMATION ESTIMATION REPRESENTATION LEARNING

Estimating Mutual Information for Discrete-Continuous Mixtures

NeurIPS 2017 alexandreguichet/MFS

We provide numerical experiments suggesting superiority of the proposed estimator compared to other heuristics of adding small continuous noise to all the samples and applying standard estimators tailored for purely continuous variables, and quantizing the samples and applying standard estimators tailored for purely discrete variables.

MUTUAL INFORMATION ESTIMATION