Search Results for author: Bernhard Schölkopf

Found 340 papers, 141 papers with code

Provable Privacy with Non-Private Pre-Processing

no code implementations19 Mar 2024 Yaxi Hu, Amartya Sanyal, Bernhard Schölkopf

When analysing Differentially Private (DP) machine learning pipelines, the potential privacy cost of data-dependent pre-processing is frequently overlooked in privacy accounting.

Imputation Quantization

Hallmarks of Optimization Trajectories in Neural Networks and LLMs: The Lengths, Bends, and Dead Ends

no code implementations12 Mar 2024 Sidak Pal Singh, Bobby He, Thomas Hofmann, Bernhard Schölkopf

We propose a fresh take on understanding the mechanisms of neural networks by analyzing the rich structure of parameters contained within their optimization trajectories.

Skill or Luck? Return Decomposition via Advantage Functions

no code implementations20 Feb 2024 Hsiao-Ru Pan, Bernhard Schölkopf

Learning from off-policy data is essential for sample-efficient reinforcement learning.

Competition of Mechanisms: Tracing How Language Models Handle Facts and Counterfactuals

1 code implementation18 Feb 2024 Francesco Ortu, Zhijing Jin, Diego Doimo, Mrinmaya Sachan, Alberto Cazzaniga, Bernhard Schölkopf

Interpretability research aims to bridge the gap between the empirical success and our scientific understanding of the inner workings of large language models (LLMs).

The Essential Role of Causality in Foundation World Models for Embodied AI

no code implementations6 Feb 2024 Tarun Gupta, Wenbo Gong, Chao Ma, Nick Pawlowski, Agrin Hilmkil, Meyer Scetbon, Ade Famoti, Ashley Juan Llorens, Jianfeng Gao, Stefan Bauer, Danica Kragic, Bernhard Schölkopf, Cheng Zhang

This paper focuses on the prospects of building foundation world models for the upcoming generation of embodied agents and presents a novel viewpoint on the significance of causality within these.

Misconceptions

A Probabilistic Model to explain Self-Supervised Representation Learning

no code implementations2 Feb 2024 Alice Bizeul, Bernhard Schölkopf, Carl Allen

Self-supervised learning (SSL) learns representations by leveraging an auxiliary unsupervised task, such as classifying semantically related samples, e. g. different data augmentations or modalities.

Representation Learning Self-Supervised Learning

Do Language Models Exhibit the Same Cognitive Biases in Problem Solving as Human Learners?

no code implementations31 Jan 2024 Andreas Opedal, Alessandro Stolfo, Haruki Shirakami, Ying Jiao, Ryan Cotterell, Bernhard Schölkopf, Abulhair Saparov, Mrinmaya Sachan

We find evidence that LLMs, with and without instruction-tuning, exhibit human-like biases in both the text-comprehension and the solution-planning steps of the solving process, but not during the final step which relies on the problem's arithmetic expressions (solution execution).

Reading Comprehension

RAVEN: Rethinking Adversarial Video Generation with Efficient Tri-plane Networks

no code implementations11 Jan 2024 Partha Ghosh, Soubhik Sanyal, Cordelia Schmid, Bernhard Schölkopf

To capture these dependencies, our approach incorporates a hybrid explicit-implicit tri-plane representation inspired by 3D-aware generative frameworks developed for three-dimensional object representation and employs a singular latent code to model an entire video sequence.

Generative Adversarial Network Optical Flow Estimation +1

Independent Mechanism Analysis and the Manifold Hypothesis

no code implementations20 Dec 2023 Shubhangi Ghosh, Luigi Gresele, Julius von Kügelgen, Michel Besserve, Bernhard Schölkopf

As typical in ICA, previous work focused on the case with an equal number of latent components and observed mixtures.

Representation Learning

Inferring Atmospheric Properties of Exoplanets with Flow Matching and Neural Importance Sampling

no code implementations13 Dec 2023 Timothy D. Gebhard, Jonas Wildberger, Maximilian Dax, Daniel Angerhausen, Sascha P. Quanz, Bernhard Schölkopf

Atmospheric retrievals (AR) characterize exoplanets by estimating atmospheric parameters from observed light spectra, typically by framing the task as a Bayesian inference problem.

Bayesian Inference

CLadder: Assessing Causal Reasoning in Language Models

1 code implementation NeurIPS 2023 Zhijing Jin, Yuen Chen, Felix Leeb, Luigi Gresele, Ojasv Kamal, Zhiheng Lyu, Kevin Blin, Fernando Gonzalez Adauto, Max Kleiman-Weiner, Mrinmaya Sachan, Bernhard Schölkopf

Much of the existing work in natural language processing (NLP) focuses on evaluating commonsense causal reasoning in LLMs, thus failing to assess whether a model can perform causal inference in accordance with a set of well-defined formal rules.

Causal Inference Commonsense Causal Reasoning +1

GraphDreamer: Compositional 3D Scene Synthesis from Scene Graphs

no code implementations30 Nov 2023 Gege Gao, Weiyang Liu, Anpei Chen, Andreas Geiger, Bernhard Schölkopf

As pretrained text-to-image diffusion models become increasingly powerful, recent efforts have been made to distill knowledge from these text-to-image pretrained models for optimizing a text-guided 3D model.

Navigating the Ocean of Biases: Political Bias Attribution in Language Models via Causal Structures

1 code implementation15 Nov 2023 David F. Jenny, Yann Billeter, Mrinmaya Sachan, Bernhard Schölkopf, Zhijing Jin

The rapid advancement of Large Language Models (LLMs) has sparked intense debate regarding their ability to perceive and interpret complex socio-political landscapes.

Decision Making

Parameter-Efficient Orthogonal Finetuning via Butterfly Factorization

1 code implementation10 Nov 2023 Weiyang Liu, Zeju Qiu, Yao Feng, Yuliang Xiu, Yuxuan Xue, Longhui Yu, Haiwen Feng, Zhen Liu, Juyeon Heo, Songyou Peng, Yandong Wen, Michael J. Black, Adrian Weller, Bernhard Schölkopf

We apply this parameterization to OFT, creating a novel parameter-efficient finetuning method, called Orthogonal Butterfly (BOFT).

Ghost on the Shell: An Expressive Representation of General 3D Shapes

no code implementations23 Oct 2023 Zhen Liu, Yao Feng, Yuliang Xiu, Weiyang Liu, Liam Paull, Michael J. Black, Bernhard Schölkopf

Recent work has focused on the former, and methods for reconstructing open surfaces do not support fast reconstruction with material and lighting or unconditional generative modelling.

Deep Backtracking Counterfactuals for Causally Compliant Explanations

no code implementations11 Oct 2023 Klaus-Rudolf Kladny, Julius von Kügelgen, Bernhard Schölkopf, Michael Muehlebach

Counterfactuals answer questions of what would have been observed under altered circumstances and can therefore offer valuable insights.

counterfactual Philosophy

Borges and AI

no code implementations27 Sep 2023 Léon Bottou, Bernhard Schölkopf

Many believe that Large Language Models (LLMs) open the era of Artificial Intelligence (AI).

Language Modelling

Parameterizing pressure-temperature profiles of exoplanet atmospheres with neural networks

1 code implementation6 Sep 2023 Timothy D. Gebhard, Daniel Angerhausen, Björn S. Konrad, Eleonora Alei, Sascha P. Quanz, Bernhard Schölkopf

When training and evaluating our method on two publicly available datasets of self-consistent PT profiles, we find that our method achieves, on average, better fit quality than existing baseline methods, despite using fewer parameters.

Bayesian Inference

SE(3) Equivariant Augmented Coupling Flows

1 code implementation NeurIPS 2023 Laurence I. Midgley, Vincent Stimper, Javier Antorán, Emile Mathieu, Bernhard Schölkopf, José Miguel Hernández-Lobato

Coupling normalizing flows allow for fast sampling and density evaluation, making them the tool of choice for probabilistic modeling of physical systems.

Benchmarking Offline Reinforcement Learning on Real-Robot Hardware

2 code implementations28 Jul 2023 Nico Gürtler, Sebastian Blaes, Pavel Kolev, Felix Widmaier, Manuel Wüthrich, Stefan Bauer, Bernhard Schölkopf, Georg Martius

To coordinate the efforts of the research community toward tackling this problem, we propose a benchmark including: i) a large collection of data for offline learning from a dexterous manipulation platform on two tasks, obtained with capable RL agents trained in simulation; ii) the option to execute learned policies on a real-world robotic system and a simulation for efficient debugging.

Benchmarking reinforcement-learning

Spuriosity Didn't Kill the Classifier: Using Invariant Predictions to Harness Spurious Features

no code implementations19 Jul 2023 Cian Eastwood, Shashank Singh, Andrei Liviu Nicolicioiu, Marin Vlastelica, Julius von Kügelgen, Bernhard Schölkopf

To avoid failures on out-of-distribution data, recent works have sought to extract features that have an invariant or stable relationship with the label across domains, discarding "spurious" or unstable features whose relationship with the label changes across domains.

The Expressive Leaky Memory Neuron: an Efficient and Expressive Phenomenological Neuron Model Can Solve Long-Horizon Tasks

1 code implementation14 Jun 2023 Aaron Spieler, Nasim Rahaman, Georg Martius, Bernhard Schölkopf, Anna Levina

Biological cortical neurons are remarkably sophisticated computational devices, temporally integrating their vast synaptic input over an intricate dendritic tree, subject to complex, nonlinearly interacting internal biological processes.

Classification Long-range modeling +3

Controlling Text-to-Image Diffusion by Orthogonal Finetuning

no code implementations NeurIPS 2023 Zeju Qiu, Weiyang Liu, Haiwen Feng, Yuxuan Xue, Yao Feng, Zhen Liu, Dan Zhang, Adrian Weller, Bernhard Schölkopf

To tackle this challenge, we introduce a principled finetuning method -- Orthogonal Finetuning (OFT), for adapting text-to-image diffusion models to downstream tasks.

Can Large Language Models Infer Causation from Correlation?

1 code implementation9 Jun 2023 Zhijing Jin, Jiarui Liu, Zhiheng Lyu, Spencer Poff, Mrinmaya Sachan, Rada Mihalcea, Mona Diab, Bernhard Schölkopf

In this work, we propose the first benchmark dataset to test the pure causal inference skills of large language models (LLMs).

Causal Inference

Causal Effect Estimation from Observational and Interventional Data Through Matrix Weighted Linear Estimators

1 code implementation9 Jun 2023 Klaus-Rudolf Kladny, Julius von Kügelgen, Bernhard Schölkopf, Michael Muehlebach

We study causal effect estimation from a mixture of observational and interventional data in a confounded linear regression model with multivariate treatments.

Stochastic Marginal Likelihood Gradients using Neural Tangent Kernels

1 code implementation6 Jun 2023 Alexander Immer, Tycho F. A. van der Ouderaa, Mark van der Wilk, Gunnar Rätsch, Bernhard Schölkopf

Recent works show that Bayesian model selection with Laplace approximations can allow to optimize such hyperparameters just like standard neural network parameters using gradients and on the training data.

Hyperparameter Optimization Model Selection

Learning Linear Causal Representations from Interventions under General Nonlinear Mixing

no code implementations NeurIPS 2023 Simon Buchholz, Goutham Rajendran, Elan Rosenfeld, Bryon Aragam, Bernhard Schölkopf, Pradeep Ravikumar

We study the problem of learning causal representations from unknown, latent interventions in a general setting, where the latent distribution is Gaussian but the mixing function is completely general.

counterfactual

Delphic Offline Reinforcement Learning under Nonidentifiable Hidden Confounding

no code implementations1 Jun 2023 Alizée Pace, Hugo Yèche, Bernhard Schölkopf, Gunnar Rätsch, Guy Tennenholtz

A prominent challenge of offline reinforcement learning (RL) is the issue of hidden confounding: unobserved variables may influence both the actions taken by the agent and the observed outcomes.

Management Offline RL +2

Membership Inference Attacks against Language Models via Neighbourhood Comparison

1 code implementation29 May 2023 Justus Mattern, FatemehSadat Mireshghallah, Zhijing Jin, Bernhard Schölkopf, Mrinmaya Sachan, Taylor Berg-Kirkpatrick

To investigate whether this fragility provides a layer of safety, we propose and evaluate neighbourhood attacks, which compare model scores for a given sample to scores of synthetically generated neighbour texts and therefore eliminate the need for access to the training data distribution.

Causal Component Analysis

1 code implementation NeurIPS 2023 Liang Wendong, Armin Kekić, Julius von Kügelgen, Simon Buchholz, Michel Besserve, Luigi Gresele, Bernhard Schölkopf

As a corollary, this interventional perspective also leads to new identifiability results for nonlinear ICA -- a special case of CauCA with an empty graph -- requiring strictly fewer datasets than previous results.

Representation Learning

Flow Matching for Scalable Simulation-Based Inference

1 code implementation NeurIPS 2023 Maximilian Dax, Jonas Wildberger, Simon Buchholz, Stephen R. Green, Jakob H. Macke, Bernhard Schölkopf

Neural posterior estimation methods based on discrete normalizing flows have become established tools for simulation-based inference (SBI), but scaling them to high-dimensional problems can be challenging.

All Roads Lead to Rome? Exploring the Invariance of Transformers' Representations

1 code implementation23 May 2023 Yuxin Ren, Qipeng Guo, Zhijing Jin, Shauli Ravfogel, Mrinmaya Sachan, Bernhard Schölkopf, Ryan Cotterell

Transformer models bring propelling advances in various NLP tasks, thus inducing lots of interpretability research on the learned representations of the models.

Provably Learning Object-Centric Representations

no code implementations23 May 2023 Jack Brady, Roland S. Zimmermann, Yash Sharma, Bernhard Schölkopf, Julius von Kügelgen, Wieland Brendel

Under this generative process, we prove that the ground-truth object representations can be identified by an invertible and compositional inference model, even in the presence of dependencies between objects.

Object Representation Learning

Estimation Beyond Data Reweighting: Kernel Method of Moments

1 code implementation18 May 2023 Heiner Kremer, Yassine Nemmour, Bernhard Schölkopf, Jia-Jie Zhu

We provide a variant of our estimator for conditional moment restrictions and show that it is asymptotically first-order optimal for such problems.

Causal Inference

The Hessian perspective into the Nature of Convolutional Neural Networks

no code implementations16 May 2023 Sidak Pal Singh, Thomas Hofmann, Bernhard Schölkopf

While Convolutional Neural Networks (CNNs) have long been investigated and applied, as well as theorized, we aim to provide a slightly different perspective into their nature -- through the perspective of their Hessian maps.

Beyond Good Intentions: Reporting the Research Landscape of NLP for Social Good

1 code implementation9 May 2023 Fernando Gonzalez, Zhijing Jin, Bernhard Schölkopf, Tom Hope, Mrinmaya Sachan, Rada Mihalcea

Using state-of-the-art NLP models, we address each of these tasks and use them on the entire ACL Anthology, resulting in a visualization workspace that gives researchers a comprehensive overview of the field of NLP4SG.

Out-of-Variable Generalization for Discriminative Models

no code implementations16 Apr 2023 Siyuan Guo, Jonas Wildberger, Bernhard Schölkopf

The ability of an agent to do well in new environments is a critical aspect of intelligence.

Out-of-Distribution Generalization

Dataflow graphs as complete causal graphs

1 code implementation16 Mar 2023 Andrei Paleyes, Siyuan Guo, Bernhard Schölkopf, Neil D. Lawrence

Component-based development is one of the core principles behind modern software engineering practices.

Generalizing and Decoupling Neural Collapse via Hyperspherical Uniformity Gap

1 code implementation11 Mar 2023 Weiyang Liu, Longhui Yu, Adrian Weller, Bernhard Schölkopf

We then use hyperspherical uniformity (which characterizes the degree of uniformity on the unit hypersphere) as a unified framework to quantify these two objectives.

Likelihood Annealing: Fast Calibrated Uncertainty for Regression

no code implementations21 Feb 2023 Uddeshya Upadhyay, Jae Myung Kim, Cordelia Schmidt, Bernhard Schölkopf, Zeynep Akata

Recent advances in deep learning have shown that uncertainty estimation is becoming increasingly important in applications such as medical imaging, natural language processing, and autonomous systems.

Denoising Image Super-Resolution +2

On the Interventional Kullback-Leibler Divergence

no code implementations10 Feb 2023 Jonas Wildberger, Siyuan Guo, Arnab Bhattacharyya, Bernhard Schölkopf

Modern machine learning approaches excel in static settings where a large amount of i. i. d.

Robustness Implies Fairness in Causal Algorithmic Recourse

2 code implementations7 Feb 2023 Ahmad-Reza Ehyaei, Amir-Hossein Karimi, Bernhard Schölkopf, Setareh Maghsudi

Algorithmic recourse aims to disclose the inner workings of the black-box decision process in situations where decisions have significant consequences, by providing recommendations to empower beneficiaries to achieve a more favorable outcome.

Adversarial Robustness Fairness

Towards fully covariant machine learning

no code implementations31 Jan 2023 Soledad Villar, David W. Hogg, Weichi Yao, George A. Kevrekidis, Bernhard Schölkopf

We discuss links to causal modeling, and argue that the implementation of passive symmetries is particularly valuable when the goal of the learning problem is to generalize out of sample.

Moûsai: Text-to-Music Generation with Long-Context Latent Diffusion

1 code implementation27 Jan 2023 Flavio Schneider, Ojasv Kamal, Zhijing Jin, Bernhard Schölkopf

Recent years have seen the rapid development of large generative models for text; however, much less research has explored the connection between text and another "language" of communication -- music.

Image Generation Music Generation +1

Multi-Armed Bandits and Quantum Channel Oracles

no code implementations20 Jan 2023 Simon Buchholz, Jonas M. Kübler, Bernhard Schölkopf

Here we introduce further bandit models where we only have limited access to the randomness of the rewards, but we can still query the arms in superposition.

Multi-Armed Bandits reinforcement-learning +1

Understanding Stereotypes in Language Models: Towards Robust Measurement and Zero-Shot Debiasing

no code implementations20 Dec 2022 Justus Mattern, Zhijing Jin, Mrinmaya Sachan, Rada Mihalcea, Bernhard Schölkopf

Generated texts from large pretrained language models have been shown to exhibit a variety of harmful, human-like biases about various demographics.

Benchmarking

Adapting to noise distribution shifts in flow-based gravitational-wave inference

no code implementations16 Nov 2022 Jonas Wildberger, Maximilian Dax, Stephen R. Green, Jonathan Gair, Michael Pürrer, Jakob H. Macke, Alessandra Buonanno, Bernhard Schölkopf

Deep learning techniques for gravitational-wave parameter estimation have emerged as a fast alternative to standard samplers $\unicode{x2013}$ producing results of comparable accuracy.

Federated Causal Discovery From Interventions

3 code implementations7 Nov 2022 Amin Abyaneh, Nino Scherrer, Patrick Schwab, Stefan Bauer, Bernhard Schölkopf, Arash Mehrjou

We propose FedCDI, a federated framework for inferring causal structures from distributed data containing interventional samples.

Causal Discovery Federated Learning +1

A General Purpose Neural Architecture for Geospatial Systems

no code implementations4 Nov 2022 Nasim Rahaman, Martin Weiss, Frederik Träuble, Francesco Locatello, Alexandre Lacoste, Yoshua Bengio, Chris Pal, Li Erran Li, Bernhard Schölkopf

Geospatial Information Systems are used by researchers and Humanitarian Assistance and Disaster Response (HADR) practitioners to support a wide variety of important applications.

Disaster Response Humanitarian +1

Iterative Teaching by Data Hallucination

1 code implementation31 Oct 2022 Zeju Qiu, Weiyang Liu, Tim Z. Xiao, Zhen Liu, Umang Bhatt, Yucen Luo, Adrian Weller, Bernhard Schölkopf

We consider the problem of iterative machine teaching, where a teacher sequentially provides examples based on the status of a learner under a discrete input space (i. e., a pool of finite samples), which greatly limits the teacher's capability.

Hallucination

Spectral Representation Learning for Conditional Moment Models

no code implementations29 Oct 2022 Ziyu Wang, Yucen Luo, Yueru Li, Jun Zhu, Bernhard Schölkopf

For nonparametric conditional moment models, efficient estimation often relies on preimposed conditions on various measures of ill-posedness of the hypothesis space, which are hard to validate when flexible models are used.

Causal Inference Representation Learning

A Causal Framework to Quantify the Robustness of Mathematical Reasoning with Language Models

1 code implementation21 Oct 2022 Alessandro Stolfo, Zhijing Jin, Kumar Shridhar, Bernhard Schölkopf, Mrinmaya Sachan

By grounding the behavioral analysis in a causal graph describing an intuitive reasoning process, we study the behavior of language models in terms of robustness and sensitivity to direct interventions in the input space.

Math Mathematical Reasoning

Neural Attentive Circuits

no code implementations14 Oct 2022 Nasim Rahaman, Martin Weiss, Francesco Locatello, Chris Pal, Yoshua Bengio, Bernhard Schölkopf, Li Erran Li, Nicolas Ballas

Recent work has seen the development of general purpose neural architectures that can be trained to perform tasks across diverse data modalities.

Point Cloud Classification text-classification +1

On the Identifiability and Estimation of Causal Location-Scale Noise Models

1 code implementation13 Oct 2022 Alexander Immer, Christoph Schultheiss, Julia E. Vogt, Bernhard Schölkopf, Peter Bühlmann, Alexander Marx

We study the class of location-scale or heteroscedastic noise models (LSNMs), in which the effect $Y$ can be written as a function of the cause $X$ and a noise source $N$ independent of $X$, which may be scaled by a positive function $g$ over the cause, i. e., $Y = f(X) + g(X)N$.

Causal Discovery Causal Inference

Neural Importance Sampling for Rapid and Reliable Gravitational-Wave Inference

1 code implementation11 Oct 2022 Maximilian Dax, Stephen R. Green, Jonathan Gair, Michael Pürrer, Jonas Wildberger, Jakob H. Macke, Alessandra Buonanno, Bernhard Schölkopf

This shows a median sample efficiency of $\approx 10\%$ (two orders-of-magnitude better than standard samplers) as well as a ten-fold reduction in the statistical uncertainty in the log evidence.

When to Make Exceptions: Exploring Language Models as Accounts of Human Moral Judgment

1 code implementation4 Oct 2022 Zhijing Jin, Sydney Levine, Fernando Gonzalez, Ojasv Kamal, Maarten Sap, Mrinmaya Sachan, Rada Mihalcea, Josh Tenenbaum, Bernhard Schölkopf

Using a state-of-the-art large language model (LLM) as a basis, we propose a novel moral chain of thought (MORALCOT) prompting strategy that combines the strengths of LLMs with theories of moral reasoning developed in cognitive science to predict human moral judgments.

Language Modelling Large Language Model +1

Function Classes for Identifiable Nonlinear Independent Component Analysis

no code implementations12 Aug 2022 Simon Buchholz, Michel Besserve, Bernhard Schölkopf

Several families of spurious solutions fitting perfectly the data, but that do not correspond to the ground truth factors can be constructed in generic settings.

Flow Annealed Importance Sampling Bootstrap

3 code implementations3 Aug 2022 Laurence Illing Midgley, Vincent Stimper, Gregor N. C. Simm, Bernhard Schölkopf, José Miguel Hernández-Lobato

Normalizing flows are tractable density models that can approximate complicated target distributions, e. g. Boltzmann distributions of physical systems.

Probable Domain Generalization via Quantile Risk Minimization

2 code implementations20 Jul 2022 Cian Eastwood, Alexander Robey, Shashank Singh, Julius von Kügelgen, Hamed Hassani, George J. Pappas, Bernhard Schölkopf

By minimizing the $\alpha$-quantile of predictor's risk distribution over domains, QRM seeks predictors that perform well with probability $\alpha$.

Domain Generalization

Structural Causal 3D Reconstruction

no code implementations20 Jul 2022 Weiyang Liu, Zhen Liu, Liam Paull, Adrian Weller, Bernhard Schölkopf

This paper considers the problem of unsupervised 3D object reconstruction from in-the-wild single-view images.

3D Object Reconstruction 3D Reconstruction +2

Assaying Out-Of-Distribution Generalization in Transfer Learning

1 code implementation19 Jul 2022 Florian Wenzel, Andrea Dittadi, Peter Vincent Gehler, Carl-Johann Simon-Gabriel, Max Horn, Dominik Zietlow, David Kernert, Chris Russell, Thomas Brox, Bernt Schiele, Bernhard Schölkopf, Francesco Locatello

Since out-of-distribution generalization is a generally ill-posed problem, various proxy targets (e. g., calibration, adversarial robustness, algorithmic corruptions, invariance across shifts) were studied across different research programs resulting in different recommendations.

Adversarial Robustness Out-of-Distribution Generalization +1

Probing the Robustness of Independent Mechanism Analysis for Representation Learning

no code implementations13 Jul 2022 Joanna Sliwa, Shubhangi Ghosh, Vincent Stimper, Luigi Gresele, Bernhard Schölkopf

One aim of representation learning is to recover the original latent code that generated the data, a task which requires additional information or inductive biases.

Representation Learning

Functional Generalized Empirical Likelihood Estimation for Conditional Moment Restrictions

1 code implementation11 Jul 2022 Heiner Kremer, Jia-Jie Zhu, Krikamol Muandet, Bernhard Schölkopf

Important problems in causal inference, economics, and, more generally, robust machine learning can be expressed as conditional moment restrictions, but estimation becomes challenging as it requires solving a continuum of unconditional moment restrictions.

BIG-bench Machine Learning Causal Inference

Variational Causal Dynamics: Discovering Modular World Models from Interventions

no code implementations22 Jun 2022 Anson Lei, Bernhard Schölkopf, Ingmar Posner

In doing so, VCD significantly extends the capabilities of the current state-of-the-art in latent world models while also comparing favourably in terms of prediction accuracy.

Causal Discovery Variational Inference

AutoML Two-Sample Test

3 code implementations17 Jun 2022 Jonas M. Kübler, Vincent Stimper, Simon Buchholz, Krikamol Muandet, Bernhard Schölkopf

Two-sample tests are important in statistics and machine learning, both as tools for scientific discovery as well as to detect distribution shifts.

AutoML Two-sample testing +1

Sampling without Replacement Leads to Faster Rates in Finite-Sum Minimax Optimization

no code implementations7 Jun 2022 Aniket Das, Bernhard Schölkopf, Michael Muehlebach

We obtain tight convergence rates for RR and SO and demonstrate that these strategies lead to faster convergence than uniform sampling.

Amortized Inference for Causal Structure Learning

1 code implementation25 May 2022 Lars Lorch, Scott Sussex, Jonas Rothfuss, Andreas Krause, Bernhard Schölkopf

Rather than searching over structures, we train a variational inference model to directly predict the causal structure from observational or interventional data.

Causal Discovery Inductive Bias +1

Original or Translated? A Causal Analysis of the Impact of Translationese on Machine Translation Performance

1 code implementation NAACL 2022 Jingwei Ni, Zhijing Jin, Markus Freitag, Mrinmaya Sachan, Bernhard Schölkopf

We show that these two factors have a large causal effect on the MT performance, in addition to the test-model direction mismatch highlighted by existing work on the impact of translationese.

Machine Translation Translation

Half-sibling regression meets exoplanet imaging: PSF modeling and subtraction using a flexible, domain knowledge-driven, causal framework

1 code implementation7 Apr 2022 Timothy D. Gebhard, Markus J. Bonse, Sascha P. Quanz, Bernhard Schölkopf

Our HSR-based method provides an alternative, flexible and promising approach to the challenge of modeling and subtracting the stellar PSF and systematic noise in exoplanet imaging data.

Denoising Pupil Tracking +1

From Statistical to Causal Learning

no code implementations1 Apr 2022 Bernhard Schölkopf, Julius von Kügelgen

We describe basic ideas underlying research to build and understand artificially intelligent systems: from symbolic approaches via statistical learning to interventional models relying on concepts of causality.

BIG-bench Machine Learning

Phenomenology of Double Descent in Finite-Width Neural Networks

no code implementations ICLR 2022 Sidak Pal Singh, Aurelien Lucchi, Thomas Hofmann, Bernhard Schölkopf

`Double descent' delineates the generalization behaviour of models depending on the regime they belong to: under- or over-parameterized.

Leveling Down in Computer Vision: Pareto Inefficiencies in Fair Deep Classifiers

no code implementations CVPR 2022 Dominik Zietlow, Michael Lohaus, Guha Balakrishnan, Matthäus Kleindessner, Francesco Locatello, Bernhard Schölkopf, Chris Russell

Algorithmic fairness is frequently motivated in terms of a trade-off in which overall performance is decreased so as to improve performance on disadvantaged groups where the algorithm would otherwise be less accurate.

Fairness

Score matching enables causal discovery of nonlinear additive noise models

no code implementations8 Mar 2022 Paul Rolland, Volkan Cevher, Matthäus Kleindessner, Chris Russel, Bernhard Schölkopf, Dominik Janzing, Francesco Locatello

This paper demonstrates how to recover causal graphs from the score of the data distribution in non-linear additive (Gaussian) noise models.

Causal Discovery

Interventions, Where and How? Experimental Design for Causal Models at Scale

1 code implementation3 Mar 2022 Panagiotis Tigas, Yashas Annadani, Andrew Jesson, Bernhard Schölkopf, Yarin Gal, Stefan Bauer

Existing methods in experimental design for causal discovery from limited data either rely on linear assumptions for the SCM or select only the intervention target.

Causal Discovery Experimental Design

Logical Fallacy Detection

2 code implementations28 Feb 2022 Zhijing Jin, Abhinav Lalwani, Tejas Vaidhya, Xiaoyu Shen, Yiwen Ding, Zhiheng Lyu, Mrinmaya Sachan, Rada Mihalcea, Bernhard Schölkopf

In this paper, we propose the task of logical fallacy detection, and provide a new dataset (Logic) of logical fallacies generally found in text, together with an additional challenge set for detecting logical fallacies in climate change claims (LogicClimate).

Language Modelling Logical Fallacies +2

Compositional Multi-Object Reinforcement Learning with Linear Relation Networks

no code implementations31 Jan 2022 Davide Mambelli, Frederik Träuble, Stefan Bauer, Bernhard Schölkopf, Francesco Locatello

Although reinforcement learning has seen remarkable progress over the last years, solving robust dexterous object-manipulation tasks in multi-object settings remains a challenge.

Object reinforcement-learning +2

On the Adversarial Robustness of Causal Algorithmic Recourse

1 code implementation21 Dec 2021 Ricardo Dominguez-Olmedo, Amir-Hossein Karimi, Bernhard Schölkopf

Algorithmic recourse seeks to provide actionable recommendations for individuals to overcome unfavorable classification outcomes from automated decision-making systems.

Adversarial Robustness Decision Making

Learning soft interventions in complex equilibrium systems

1 code implementation10 Dec 2021 Michel Besserve, Bernhard Schölkopf

Complex systems often contain feedback loops that can be described as cyclic causal models.

Towards Principled Disentanglement for Domain Generalization

1 code implementation CVPR 2022 HANLIN ZHANG, Yi-Fan Zhang, Weiyang Liu, Adrian Weller, Bernhard Schölkopf, Eric P. Xing

To tackle this challenge, we first formalize the OOD generalization problem as constrained optimization, called Disentanglement-constrained Domain Generalization (DDG).

Disentanglement Domain Generalization

Group equivariant neural posterior estimation

1 code implementation ICLR 2022 Maximilian Dax, Stephen R. Green, Jonathan Gair, Michael Deistler, Bernhard Schölkopf, Jakob H. Macke

We here describe an alternative method to incorporate equivariances under joint transformations of parameters and data.

Cause-effect inference through spectral independence in linear dynamical systems: theoretical foundations

no code implementations29 Oct 2021 Michel Besserve, Naji Shajarisales, Dominik Janzing, Bernhard Schölkopf

A new perspective has been provided based on the principle of Independence of Causal Mechanisms (ICM), leading to the Spectral Independence Criterion (SIC), postulating that the power spectral density (PSD) of the cause time series is uncorrelated with the squared modulus of the frequency response of the filter generating the effect.

Causal Discovery Causal Inference +2

GalilAI: Out-of-Task Distribution Detection using Causal Active Experimentation for Safe Transfer RL

no code implementations29 Oct 2021 Sumedh A Sontakke, Stephen Iota, Zizhao Hu, Arash Mehrjou, Laurent Itti, Bernhard Schölkopf

Extending the successes in supervised learning methods to the reinforcement learning (RL) setting, however, is difficult due to the data generating process - RL agents actively query their environment for data, and the data are a function of the policy followed by the agent.

Out of Distribution (OOD) Detection Reinforcement Learning (RL)

Resampling Base Distributions of Normalizing Flows

1 code implementation29 Oct 2021 Vincent Stimper, Bernhard Schölkopf, José Miguel Hernández-Lobato

Normalizing flows are a popular class of models for approximating probability distributions.

Ranked #47 on Image Generation on CIFAR-10 (bits/dimension metric)

Density Estimation Image Generation

Iterative Teaching by Label Synthesis

no code implementations NeurIPS 2021 Weiyang Liu, Zhen Liu, Hanchen Wang, Liam Paull, Bernhard Schölkopf, Adrian Weller

In this paper, we consider the problem of iterative machine teaching, where a teacher provides examples sequentially based on the current iterative learner.

Action-Sufficient State Representation Learning for Control with Structural Constraints

no code implementations12 Oct 2021 Biwei Huang, Chaochao Lu, Liu Leqi, José Miguel Hernández-Lobato, Clark Glymour, Bernhard Schölkopf, Kun Zhang

Perceived signals in real-world scenarios are usually high-dimensional and noisy, and finding and using their representation that contains essential and sufficient information required by downstream decision-making tasks will help improve computational efficiency and generalization ability in the tasks.

Computational Efficiency Decision Making +1

You Mostly Walk Alone: Analyzing Feature Attribution in Trajectory Prediction

no code implementations ICLR 2022 Osama Makansi, Julius von Kügelgen, Francesco Locatello, Peter Gehler, Dominik Janzing, Thomas Brox, Bernhard Schölkopf

Applying this procedure to state-of-the-art trajectory prediction methods on standard benchmark datasets shows that they are, in fact, unable to reason about interactions.

Attribute Trajectory Prediction

Causal Direction of Data Collection Matters: Implications of Causal and Anticausal Learning for NLP

1 code implementation EMNLP 2021 Zhijing Jin, Julius von Kügelgen, Jingwei Ni, Tejas Vaidhya, Ayush Kaushal, Mrinmaya Sachan, Bernhard Schölkopf

The principle of independent causal mechanisms (ICM) states that generative processes of real world data consist of independent modules which do not influence or inform each other.

Causal Inference Domain Adaptation

Spatial Context Awareness for Unsupervised Change Detection in Optical Satellite Images

no code implementations5 Oct 2021 Lukas Kondmann, Aysim Toker, Sudipan Saha, Bernhard Schölkopf, Laura Leal-Taixé, Xiao Xiang Zhu

It uses this model to analyze differences in the pixel and its spatial context-based predictions in subsequent time periods for change detection.

Change Detection

On the interventional consistency of autoencoders

no code implementations29 Sep 2021 Giulia Lanzillotta, Felix Leeb, Stefan Bauer, Bernhard Schölkopf

Autoencoders have played a crucial role in the field of representation learning since its inception, proving to be a flexible learning scheme able to accommodate various notions of optimality of the representation.

Disentanglement

Direct Advantage Estimation

1 code implementation13 Sep 2021 Hsiao-Ru Pan, Nico Gürtler, Alexander Neitz, Bernhard Schölkopf

The predominant approach in reinforcement learning is to assign credit to actions based on the expected return.

Visual Representation Learning Does Not Generalize Strongly Within the Same Domain

1 code implementation ICLR 2022 Lukas Schott, Julius von Kügelgen, Frederik Träuble, Peter Gehler, Chris Russell, Matthias Bethge, Bernhard Schölkopf, Francesco Locatello, Wieland Brendel

An important component for generalization in machine learning is to uncover underlying latent factors of variation as well as the mechanism through which each factor acts in the world.

Representation Learning

The Role of Pretrained Representations for the OOD Generalization of Reinforcement Learning Agents

no code implementations ICLR 2022 Andrea Dittadi, Frederik Träuble, Manuel Wüthrich, Felix Widmaier, Peter Gehler, Ole Winther, Francesco Locatello, Olivier Bachem, Bernhard Schölkopf, Stefan Bauer

By training 240 representations and over 10, 000 reinforcement learning (RL) policies on a simulated robotic setup, we evaluate to what extent different properties of pretrained VAE-based representations affect the OOD generalization of downstream agents.

Reinforcement Learning (RL) Representation Learning

Source-Free Adaptation to Measurement Shift via Bottom-Up Feature Restoration

1 code implementation ICLR 2022 Cian Eastwood, Ian Mason, Christopher K. I. Williams, Bernhard Schölkopf

Existing methods for SFDA leverage entropy-minimization techniques which: (i) apply only to classification; (ii) destroy model calibration; and (iii) rely on the source model achieving a good level of feature-space class-separation in the target domain.

Source-Free Domain Adaptation

Generalization and Robustness Implications in Object-Centric Learning

1 code implementation1 Jul 2021 Andrea Dittadi, Samuele Papa, Michele De Vita, Bernhard Schölkopf, Ole Winther, Francesco Locatello

The idea behind object-centric representation learning is that natural scenes can better be modeled as compositions of objects and their relations as opposed to distributed representations.

Inductive Bias Object +3

Exploring the Latent Space of Autoencoders with Interventional Assays

1 code implementation30 Jun 2021 Felix Leeb, Stefan Bauer, Michel Besserve, Bernhard Schölkopf

Autoencoders exhibit impressive abilities to embed the data manifold into a low-dimensional latent space, making them a staple of representation learning methods.

Disentanglement

Shallow Representation is Deep: Learning Uncertainty-aware and Worst-case Random Feature Dynamics

no code implementations24 Jun 2021 Diego Agudelo-España, Yassine Nemmour, Bernhard Schölkopf, Jia-Jie Zhu

Random features is a powerful universal function approximator that inherits the theoretical rigor of kernel methods and can scale up to modern learning tasks.

Real-time gravitational-wave science with neural posterior estimation

1 code implementation23 Jun 2021 Maximilian Dax, Stephen R. Green, Jonathan Gair, Jakob H. Macke, Alessandra Buonanno, Bernhard Schölkopf

We demonstrate unprecedented accuracy for rapid gravitational-wave parameter estimation with deep learning.

Algorithmic Recourse in Partially and Fully Confounded Settings Through Bounding Counterfactual Effects

no code implementations22 Jun 2021 Julius von Kügelgen, Nikita Agarwal, Jakob Zeitler, Afsaneh Mastouri, Bernhard Schölkopf

Algorithmic recourse aims to provide actionable recommendations to individuals to obtain a more favourable outcome from an automated decision-making system.

counterfactual Decision Making

The Inductive Bias of Quantum Kernels

1 code implementation NeurIPS 2021 Jonas M. Kübler, Simon Buchholz, Bernhard Schölkopf

Quantum computers offer the possibility to efficiently compute inner products of exponentially large density operators that are classically hard to compute.

Inductive Bias Quantum Machine Learning

Instrument Space Selection for Kernel Maximum Moment Restriction

1 code implementation7 Jun 2021 Rui Zhang, Krikamol Muandet, Bernhard Schölkopf, Masaaki Imaizumi

Kernel maximum moment restriction (KMMR) recently emerges as a popular framework for instrumental variable (IV) based conditional moment restriction (CMR) models with important applications in conditional moment (CM) testing and parameter estimation for IV regression and proximal causal learning.

Diffusion-Based Representation Learning

no code implementations29 May 2021 Korbinian Abstreiter, Sarthak Mittal, Stefan Bauer, Bernhard Schölkopf, Arash Mehrjou

In contrast, the introduced diffusion-based representation learning relies on a new formulation of the denoising score matching objective and thus encodes the information needed for denoising.

Denoising Representation Learning +1

DiBS: Differentiable Bayesian Structure Learning

2 code implementations NeurIPS 2021 Lars Lorch, Jonas Rothfuss, Bernhard Schölkopf, Andreas Krause

In this work, we propose a general, fully differentiable framework for Bayesian structure learning (DiBS) that operates in the continuous space of a latent probabilistic graph representation.

Causal Discovery Variational Inference

Fast and Slow Learning of Recurrent Independent Mechanisms

no code implementations18 May 2021 Kanika Madan, Nan Rosemary Ke, Anirudh Goyal, Bernhard Schölkopf, Yoshua Bengio

To study these ideas, we propose a particular training framework in which we assume that the pieces of knowledge an agent needs and its reward function are stationary and can be re-used across tasks.

Meta-Learning

Regret Bounds for Gaussian-Process Optimization in Large Domains

1 code implementation NeurIPS 2021 Manuel Wüthrich, Bernhard Schölkopf, Andreas Krause

These regret bounds illuminate the relationship between the number of evaluations, the domain size (i. e. cardinality of finite domains / Lipschitz constant of the covariance function in continuous domains), and the optimality of the retrieved function value.

Pyfectious: An individual-level simulator to discover optimal containment polices for epidemic diseases

1 code implementation24 Mar 2021 Arash Mehrjou, Ashkan Soleymani, Amin Abyaneh, Samir Bhatt, Bernhard Schölkopf, Stefan Bauer

Simulating the spread of infectious diseases in human communities is critical for predicting the trajectory of an epidemic and verifying various policies to control the devastating impacts of the outbreak.

A prior-based approximate latent Riemannian metric

no code implementations9 Mar 2021 Georgios Arvanitidis, Bogdan Georgiev, Bernhard Schölkopf

In this work we propose a surrogate conformal Riemannian metric in the latent space of a generative model that is simple, efficient and robust.

Learning with Hyperspherical Uniformity

1 code implementation2 Mar 2021 Weiyang Liu, Rongmei Lin, Zhen Liu, Li Xiong, Bernhard Schölkopf, Adrian Weller

Due to the over-parameterization nature, neural networks are a powerful tool for nonlinear function approximation.

Inductive Bias L2 Regularization

Nonlinear Invariant Risk Minimization: A Causal Approach

no code implementations24 Feb 2021 Chaochao Lu, Yuhuai Wu, Jośe Miguel Hernández-Lobato, Bernhard Schölkopf

Finally, in the discussion, we further explore the aforementioned assumption and propose a more general hypothesis, called the Agnostic Hypothesis: there exist a set of hidden causal factors affecting both inputs and outcomes.

BIG-bench Machine Learning Representation Learning

Finding Stable Matchings in PhD Markets with Consistent Preferences and Cooperative Partners

no code implementations23 Feb 2021 Maximilian Mordig, Riccardo Della Vecchia, Nicolò Cesa-Bianchi, Bernhard Schölkopf

Our setting is motivated by a PhD market of students, advisors, and co-advisors, and can be generalized to supply chain networks viewed as $n$-sided markets.

Computer Science and Game Theory Theoretical Economics Combinatorics

Adversarially Robust Kernel Smoothing

1 code implementation16 Feb 2021 Jia-Jie Zhu, Christina Kouridi, Yassine Nemmour, Bernhard Schölkopf

We propose a scalable robust learning algorithm combining kernel smoothing and robust optimization.

BIG-bench Machine Learning

Conditional Distributional Treatment Effect with Kernel Conditional Mean Embeddings and U-Statistic Regression

no code implementations16 Feb 2021 Junhyung Park, Uri Shalit, Bernhard Schölkopf, Krikamol Muandet

We propose to analyse the conditional distributional treatment effect (CoDiTE), which, in contrast to the more common conditional average treatment effect (CATE), is designed to encode a treatment's distributional aspects beyond the mean.

regression

Bayesian Quadrature on Riemannian Data Manifolds

1 code implementation12 Feb 2021 Christian Fröhlich, Alexandra Gessner, Philipp Hennig, Bernhard Schölkopf, Georgios Arvanitidis

Riemannian manifolds provide a principled way to model nonlinear geometric structure inherent in data.

A Witness Two-Sample Test

1 code implementation10 Feb 2021 Jonas M. Kübler, Wittawat Jitkrittum, Bernhard Schölkopf, Krikamol Muandet

That is, the test set is used to simultaneously estimate the expectations and define the basis points, while the training set only serves to select the kernel and is discarded.

Two-sample testing Vocal Bursts Valence Prediction

Invariant Causal Representation Learning

no code implementations1 Jan 2021 Chaochao Lu, Yuhuai Wu, José Miguel Hernández-Lobato, Bernhard Schölkopf

As an alternative, we propose Invariant Causal Representation Learning (ICRL), a learning paradigm that enables out-of-distribution generalization in the nonlinear setting (i. e., nonlinear representations and nonlinear classifiers).

Out-of-Distribution Generalization Representation Learning

Spatially Structured Recurrent Modules

no code implementations ICLR 2021 Nasim Rahaman, Anirudh Goyal, Muhammad Waleed Gondal, Manuel Wuthrich, Stefan Bauer, Yash Sharma, Yoshua Bengio, Bernhard Schölkopf

Capturing the structure of a data-generating process by means of appropriate inductive biases can help in learning models that generalise well and are robust to changes in the input distribution.

Starcraft II Video Prediction

Dependency Structure Discovery from Interventions

no code implementations1 Jan 2021 Nan Rosemary Ke, Olexa Bilaniuk, Anirudh Goyal, Stefan Bauer, Bernhard Schölkopf, Michael Curtis Mozer, Hugo Larochelle, Christopher Pal, Yoshua Bengio

Promising results have driven a recent surge of interest in continuous optimization methods for Bayesian network structure learning from observational data.

Learned residual Gerchberg-Saxton network for computer generated holography

no code implementations1 Jan 2021 Lennart Schlieder, Heiner Kremer, Valentin Volchkov, Kai Melde, Peer Fischer, Bernhard Schölkopf

Instead of an iterative optimization algorithm that converges to a (sub-)optimal solution, the inverse problem can be solved by training a neural network to directly estimate the inverse operator.

Learning to interpret trajectories

no code implementations ICLR 2021 Alexander Neitz, Giambattista Parascandolo, Bernhard Schölkopf

By learning to predict trajectories of dynamical systems, model-based methods can make extensive use of all observations from past experience.

Assaying Large-scale Testing Models to Interpret COVID-19 Case Numbers

no code implementations3 Dec 2020 Michel Besserve, Simon Buchholz, Bernhard Schölkopf

Large-scale testing is considered key to assess the state of the current COVID-19 pandemic.

Applications Populations and Evolution

Causal analysis of Covid-19 Spread in Germany

no code implementations NeurIPS 2020 Atalanti Mastakouri, Bernhard Schölkopf

In this work, we study the causal relations among German regions in terms of the spread of Covid-19 since the beginning of the pandemic, taking into account the restriction policies that were applied by the different federal states.

feature selection Time Series +1

A Sober Look at the Unsupervised Learning of Disentangled Representations and their Evaluation

no code implementations27 Oct 2020 Francesco Locatello, Stefan Bauer, Mario Lucic, Gunnar Rätsch, Sylvain Gelly, Bernhard Schölkopf, Olivier Bachem

The idea behind the \emph{unsupervised} learning of \emph{disentangled} representations is that real-world data is generated by a few explanatory factors of variation which can be recovered by unsupervised learning algorithms.

Disentanglement

On the Transfer of Disentangled Representations in Realistic Settings

no code implementations ICLR 2021 Andrea Dittadi, Frederik Träuble, Francesco Locatello, Manuel Wüthrich, Vaibhav Agrawal, Ole Winther, Stefan Bauer, Bernhard Schölkopf

Learning meaningful representations that disentangle the underlying structure of the data generating process is considered to be of key importance in machine learning.

Disentanglement

Predicting Infectiousness for Proactive Contact Tracing

1 code implementation ICLR 2021 Yoshua Bengio, Prateek Gupta, Tegan Maharaj, Nasim Rahaman, Martin Weiss, Tristan Deleu, Eilif Muller, Meng Qu, Victor Schmidt, Pierre-Luc St-Charles, Hannah Alsdurf, Olexa Bilanuik, David Buckeridge, Gáetan Marceau Caron, Pierre-Luc Carrier, Joumana Ghosn, Satya Ortiz-Gagne, Chris Pal, Irina Rish, Bernhard Schölkopf, Abhinav Sharma, Jian Tang, Andrew Williams

Predictions are used to provide personalized recommendations to the individual via an app, as well as to send anonymized messages to the individual's contacts, who use this information to better predict their own infectiousness, an approach we call proactive contact tracing (PCT).

Instrumental Variable Regression via Kernel Maximum Moment Loss

2 code implementations15 Oct 2020 Rui Zhang, Masaaki Imaizumi, Bernhard Schölkopf, Krikamol Muandet

We investigate a simple objective for nonlinear instrumental variable (IV) regression based on a kernelized conditional moment restriction (CMR) known as a maximum moment restriction (MMR).

regression

Function Contrastive Learning of Transferable Meta-Representations

no code implementations14 Oct 2020 Muhammad Waleed Gondal, Shruti Joshi, Nasim Rahaman, Stefan Bauer, Manuel Wüthrich, Bernhard Schölkopf

This \emph{meta-representation}, which is computed from a few observed examples of the underlying function, is learned jointly with the predictive model.

Contrastive Learning Few-Shot Learning

Physically constrained causal noise models for high-contrast imaging of exoplanets

no code implementations12 Oct 2020 Timothy D. Gebhard, Markus J. Bonse, Sascha P. Quanz, Bernhard Schölkopf

The detection of exoplanets in high-contrast imaging (HCI) data hinges on post-processing methods to remove spurious light from the host star.

Vocal Bursts Intensity Prediction

A survey of algorithmic recourse: definitions, formulations, solutions, and prospects

no code implementations8 Oct 2020 Amir-Hossein Karimi, Gilles Barthe, Bernhard Schölkopf, Isabel Valera

Machine learning is increasingly used to inform decision-making in sensitive situations where decisions have consequential effects on individuals' lives.

Decision Making Fairness

Causal Curiosity: RL Agents Discovering Self-supervised Experiments for Causal Representation Learning

1 code implementation7 Oct 2020 Sumedh A. Sontakke, Arash Mehrjou, Laurent Itti, Bernhard Schölkopf

Inspired by this, we attempt to equip reinforcement learning agents with the ability to perform experiments that facilitate a categorization of the rolled-out trajectories, and to subsequently infer the causal factors of the environment in a hierarchical manner.

Representation Learning Zero-Shot Learning

Function Contrastive Learning of Transferable Representations

no code implementations28 Sep 2020 Muhammad Waleed Gondal, Shruti Joshi, Nasim Rahaman, Stefan Bauer, Manuel Wuthrich, Bernhard Schölkopf

Few-shot-learning seeks to find models that are capable of fast-adaptation to novel tasks which are not encountered during training.

Contrastive Learning Few-Shot Learning

Learning explanations that are hard to vary

3 code implementations ICLR 2021 Giambattista Parascandolo, Alexander Neitz, Antonio Orvieto, Luigi Gresele, Bernhard Schölkopf

In this paper, we investigate the principle that `good explanations are hard to vary' in the context of deep learning.

Memorization

Real-time Prediction of COVID-19 related Mortality using Electronic Health Records

no code implementations31 Aug 2020 Patrick Schwab, Arash Mehrjou, Sonali Parbhoo, Leo Anthony Celi, Jürgen Hetzel, Markus Hofer, Bernhard Schölkopf, Stefan Bauer

Coronavirus Disease 2019 (COVID-19) is an emerging respiratory disease caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) with rapid human-to-human transmission and a high case fatality rate particularly in older patients.

Specificity

Learning Dynamical Systems using Local Stability Priors

no code implementations23 Aug 2020 Arash Mehrjou, Andrea Iannelli, Bernhard Schölkopf

A coupled computational approach to simultaneously learn a vector field and the region of attraction of an equilibrium point from generated trajectories of the system is proposed.

Geometrically Enriched Latent Spaces

no code implementations2 Aug 2020 Georgios Arvanitidis, Søren Hauberg, Bernhard Schölkopf

A common assumption in generative models is that the generator immerses the latent space into a Euclidean ambient space.

A Commentary on the Unsupervised Learning of Disentangled Representations

no code implementations28 Jul 2020 Francesco Locatello, Stefan Bauer, Mario Lucic, Gunnar Rätsch, Sylvain Gelly, Bernhard Schölkopf, Olivier Bachem

The goal of the unsupervised learning of disentangled representations is to separate the independent explanatory factors of variation in the data without access to supervision.

S2RMs: Spatially Structured Recurrent Modules

no code implementations13 Jul 2020 Nasim Rahaman, Anirudh Goyal, Muhammad Waleed Gondal, Manuel Wuthrich, Stefan Bauer, Yash Sharma, Yoshua Bengio, Bernhard Schölkopf

Capturing the structure of a data-generating process by means of appropriate inductive biases can help in learning models that generalize well and are robust to changes in the input distribution.

Starcraft II Video Prediction

Causal Feature Selection via Orthogonal Search

no code implementations6 Jul 2020 Ashkan Soleymani, Anant Raj, Stefan Bauer, Bernhard Schölkopf, Michel Besserve

The problem of inferring the direct causal parents of a response variable among a large set of explanatory variables is of high practical importance in many disciplines.

Causal Discovery feature selection

Metrizing Weak Convergence with Maximum Mean Discrepancies

no code implementations16 Jun 2020 Carl-Johann Simon-Gabriel, Alessandro Barp, Bernhard Schölkopf, Lester Mackey

More precisely, we prove that, on a locally compact, non-compact, Hausdorff space, the MMD of a bounded continuous Borel measurable kernel k, whose reproducing kernel Hilbert space (RKHS) functions vanish at infinity, metrizes the weak convergence of probability measures if and only if k is continuous and integrally strictly positive definite (i. s. p. d.)

Structure by Architecture: Structured Representations without Regularization

no code implementations14 Jun 2020 Felix Leeb, Guilia Lanzillotta, Yashas Annadani, Michel Besserve, Stefan Bauer, Bernhard Schölkopf

We study the problem of self-supervised structured representation learning using autoencoders for downstream tasks such as generative modeling.

Disentanglement

Kernel Distributionally Robust Optimization

2 code implementations12 Jun 2020 Jia-Jie Zhu, Wittawat Jitkrittum, Moritz Diehl, Bernhard Schölkopf

We prove a theorem that generalizes the classical duality in the mathematical problem of moments.

Stochastic Optimization

Algorithmic recourse under imperfect causal knowledge: a probabilistic approach

1 code implementation NeurIPS 2020 Amir-Hossein Karimi, Julius von Kügelgen, Bernhard Schölkopf, Isabel Valera

Recent work has discussed the limitations of counterfactual explanations to recommend actions for algorithmic recourse, and argued for the need of taking causal relationships between features into consideration.

counterfactual

Learning to Play Table Tennis From Scratch using Muscular Robots

no code implementations10 Jun 2020 Dieter Büchler, Simon Guist, Roberto Calandra, Vincent Berenz, Bernhard Schölkopf, Jan Peters

This work is the first to (a) fail-safe learn of a safety-critical dynamic task using anthropomorphic robot arms, (b) learn a precision-demanding problem with a PAM-driven system despite the control challenges and (c) train robots to play table tennis without real balls.

reinforcement-learning Reinforcement Learning (RL)

Neural Lyapunov Redesign

1 code implementation6 Jun 2020 Arash Mehrjou, Mohammad Ghavamzadeh, Bernhard Schölkopf

We provide theoretical results on the class of systems that can be treated with the proposed algorithm and empirically evaluate the effectiveness of our method using an exemplary dynamical system.

Learning Kernel Tests Without Data Splitting

1 code implementation NeurIPS 2020 Jonas M. Kübler, Wittawat Jitkrittum, Bernhard Schölkopf, Krikamol Muandet

Modern large-scale kernel-based tests such as maximum mean discrepancy (MMD) and kernelized Stein discrepancy (KSD) optimize kernel hyperparameters on a held-out sample via data splitting to obtain the most powerful test statistics.

A machine learning route between band mapping and band structure

1 code implementation20 May 2020 Rui Patrick Xian, Vincent Stimper, Marios Zacharias, Shuo Dong, Maciej Dendzik, Samuel Beaulieu, Bernhard Schölkopf, Martin Wolf, Laurenz Rettig, Christian Carbogno, Stefan Bauer, Ralph Ernstorfer

Electronic band structure (BS) and crystal structure are the two complementary identifiers of solid state materials.

Data Analysis, Statistics and Probability Materials Science Computational Physics

Necessary and sufficient conditions for causal feature selection in time series with latent common causes

no code implementations18 May 2020 Atalanti A. Mastakouri, Bernhard Schölkopf, Dominik Janzing

We study the identification of direct and indirect causes on time series and provide conditions in the presence of latent variables, which we prove to be necessary and sufficient under some graph constraints.

feature selection Time Series +1

Simpson's paradox in Covid-19 case fatality rates: a mediation analysis of age-related causal effects

1 code implementation14 May 2020 Julius von Kügelgen, Luigi Gresele, Bernhard Schölkopf

We point out limitations and extensions for future work, and, finally, discuss the role of causal reasoning in the broader context of using AI to combat the Covid-19 pandemic.

Applications Methodology

Crackovid: Optimizing Group Testing

no code implementations13 May 2020 Louis Abraham, Gary Bécigneul, Bernhard Schölkopf

We study the problem usually referred to as group testing in the context of COVID-19.

Disentangling Factors of Variations Using Few Labels

no code implementations ICLR Workshop LLD 2019 Francesco Locatello, Michael Tschannen, Stefan Bauer, Gunnar Rätsch, Bernhard Schölkopf, Olivier Bachem

Recently, Locatello et al. (2019) demonstrated that unsupervised disentanglement learning without inductive biases is theoretically impossible and that existing inductive biases and unsupervised methods do not allow to consistently learn disentangled representations.

Disentanglement Model Selection

Towards causal generative scene models via competition of experts

no code implementations27 Apr 2020 Julius von Kügelgen, Ivan Ustyuzhaninov, Peter Gehler, Matthias Bethge, Bernhard Schölkopf

Learning how to model complex scenes in a modular way with recombinable components is a pre-requisite for higher-order reasoning and acting in the physical world.

Inductive Bias Object

Quantifying the Effects of Contact Tracing, Testing, and Containment Measures in the Presence of Infection Hotspots

2 code implementations15 Apr 2020 Lars Lorch, Heiner Kremer, William Trouleau, Stratis Tsirtsis, Aron Szanto, Bernhard Schölkopf, Manuel Gomez-Rodriguez

Multiple lines of evidence strongly suggest that infection hotspots, where a single individual infects many others, play a key role in the transmission dynamics of COVID-19.

Bayesian Optimization Point Processes

A theory of independent mechanisms for extrapolation in generative models

no code implementations1 Apr 2020 Michel Besserve, Rémy Sun, Dominik Janzing, Bernhard Schölkopf

Generative models can be trained to emulate complex empirical data, but are they useful to make predictions in the context of previously unobserved environments?

Cannot find the paper you are looking for? You can Submit a new open access paper.