no code implementations • 7 Dec 2023 • Peter Bjørn Jørgensen, Jonas Busk, Ole Winther, Mikkel N. Schmidt
In machine learning energy potentials for atomic systems, forces are commonly obtained as the negative derivative of the energy function with respect to atomic positions.
1 code implementation • 21 Nov 2023 • Frederikke Isa Marin, Felix Teufel, Marc Horlacher, Dennis Madsen, Dennis Pultz, Ole Winther, Wouter Boomsma
The genome sequence contains the blueprint for governing cellular processes.
1 code implementation • 30 Oct 2023 • Beatrix M. G. Nielsen, Anders Christensen, Andrea Dittadi, Ole Winther
Diffusion models may be viewed as hierarchical variational autoencoders (VAEs) with two improvements: parameter sharing for the conditional distributions in the generative process and efficient computation of the loss as independent terms over the hierarchy.
1 code implementation • ICCV 2023 • Anders Christensen, Massimiliano Mancini, A. Sophia Koepke, Ole Winther, Zeynep Akata
We achieve this with our proposed Image-free Classifier Injection with Semantics (ICIS) that injects classifiers for new, unseen classes into pre-trained classification models in a post-hoc fashion without relying on image data.
1 code implementation • 20 Jul 2023 • Leander Girrbach, Anders Christensen, Ole Winther, Zeynep Akata, A. Sophia Koepke
Whilst this captures useful information for linear classifiers, we find that no relevant spatial structure is present in later layers of deep neural networks, making neural persistence roughly equivalent to the variance of weights.
no code implementations • 29 May 2023 • Mathias Schreiner, Ole Winther, Simon Olsson
Computing properties of molecular systems rely on estimating expectations of the (unnormalized) Boltzmann distribution.
no code implementations • 10 May 2023 • Jonas Busk, Mikkel N. Schmidt, Ole Winther, Tejs Vegge, Peter Bjørn Jørgensen
The proposed method considers both epistemic and aleatoric uncertainty and the total uncertainties are recalibrated post hoc using a nonlinear scaling function to achieve good calibration on previously unseen data, without loss of predictive accuracy.
1 code implementation • 23 Feb 2023 • Raluca Jalaboi, Ole Winther, Alfiia Galimzianova
We pre-trained all architectures on an clinical skin disease dataset, and fine-tuned them on a DermXDB subset.
1 code implementation • 29 Jan 2023 • Dimitrios Christofidellis, Giorgio Giannone, Jannis Born, Ole Winther, Teodoro Laino, Matteo Manica
Here, we propose the first multi-domain, multi-task language model that can solve a wide range of tasks in both the chemical and natural language domains.
Ranked #3 on Molecule Captioning on ChEBI-20
1 code implementation • 27 Jan 2023 • Simon Ott, Konstantin Hebenstreit, Valentin Liévin, Christoffer Egeberg Hother, Milad Moradi, Maximilian Mayrhauser, Robert Praas, Ole Winther, Matthias Samwald
Large language models (LLMs) such as GPT-4 have recently demonstrated impressive results across a wide range of tasks.
2 code implementations • 23 Sep 2022 • Valentin Liévin, Andreas Geert Motzfeldt, Ida Riis Jensen, Ole Winther
Retrieval-augmented models have proven to be effective in natural language processing tasks, yet there remains a lack of research on their optimization using variational inference.
Ranked #4 on Multiple Choice Question Answering (MCQA) on MedMCQA
no code implementations • 10 Sep 2022 • Raluca Jalaboi, Ole Winther, Alfiia Galimzianova
For poor image quality explanations, our method obtains F1-scores of between 0. 37 +- 0. 01 and 0. 70 +- 0. 01, similar to the inter-rater pairwise F1-score of between 0. 24 +- 0. 15 and 0. 83 +- 0. 06.
no code implementations • 25 Jul 2022 • Mathias Schreiner, Arghya Bhowmik, Tejs Vegge, Jonas Busk, Ole Winther
In this work, we present the dataset Transition1x containing 9. 6 million Density Functional Theory (DFT) calculations of forces and energies of molecular configurations on and around reaction pathways at the wB97x/6-31G(d) level of theory.
no code implementations • 20 Jul 2022 • Mathias Schreiner, Arghya Bhowmik, Tejs Vegge, Peter Bjørn Jørgensen, Ole Winther
We also compare with and outperform Density Functional based Tight Binding (DFTB) on both accuracy and computational resource.
1 code implementation • 17 Jul 2022 • Valentin Liévin, Christoffer Egeberg Hother, Andreas Geert Motzfeldt, Ole Winther
Although large language models (LLMs) often produce impressive outputs, it remains unclear how they perform in real-world scenarios requiring strong reasoning skills and expert domain knowledge.
Ranked #5 on Question Answering on PubMedQA
Multiple-choice Multiple Choice Question Answering (MCQA) +3
1 code implementation • 30 May 2022 • Giorgio Giannone, Didrik Nielsen, Ole Winther
At test time, the model is able to generate samples from previously unseen classes conditioned on as few as 5 samples from that class.
no code implementations • 18 Apr 2022 • Samuele Papa, Ole Winther, Andrea Dittadi
Understanding which inductive biases could be helpful for the unsupervised learning of object-centric representations of natural scenes is challenging.
no code implementations • 17 Mar 2022 • Darius Chira, Ilian Haralampiev, Ole Winther, Andrea Dittadi, Valentin Liévin
Image super-resolution (SR) techniques are used to generate a high-resolution image from a low-resolution image.
1 code implementation • 14 Feb 2022 • Raluca Jalaboi, Frederik Faye, Mauricio Orbes-Arteaga, Dan Jørgensen, Ole Winther, Alfiia Galimzianova
We assess the explanation performance in terms of identification and localization by comparing model-selected with dermatologist-selected explanations, and gradient-weighted class-activation maps with dermatologist explanation maps, respectively.
1 code implementation • 23 Oct 2021 • Giorgio Giannone, Ole Winther
In few-shot learning the model is trained on data from many sets from distributions sharing some underlying properties such as sets of characters from different alphabets or objects from different categories.
no code implementations • 13 Jul 2021 • Jonas Busk, Peter Bjørn Jørgensen, Arghya Bhowmik, Mikkel N. Schmidt, Ole Winther, Tejs Vegge
In this work we extend a message passing neural network designed specifically for predicting properties of molecules and materials with a calibrated probabilistic predictive distribution.
no code implementations • ICLR 2022 • Andrea Dittadi, Frederik Träuble, Manuel Wüthrich, Felix Widmaier, Peter Gehler, Ole Winther, Francesco Locatello, Olivier Bachem, Bernhard Schölkopf, Stefan Bauer
By training 240 representations and over 10, 000 reinforcement learning (RL) policies on a simulated robotic setup, we evaluate to what extent different properties of pretrained VAE-based representations affect the OOD generalization of downstream agents.
1 code implementation • 1 Jul 2021 • Andrea Dittadi, Samuele Papa, Michele De Vita, Bernhard Schölkopf, Ole Winther, Francesco Locatello
The idea behind object-centric representation learning is that natural scenes can better be modeled as compositions of objects and their relations as opposed to distributed representations.
no code implementations • ICML Workshop URL 2021 • Frederik Träuble, Andrea Dittadi, Manuel Wuthrich, Felix Widmaier, Peter Vincent Gehler, Ole Winther, Francesco Locatello, Olivier Bachem, Bernhard Schölkopf, Stefan Bauer
Learning data representations that are useful for various downstream tasks is a cornerstone of artificial intelligence.
Out-of-Distribution Generalization reinforcement-learning +2
1 code implementation • NeurIPS 2020 • Valentin Liévin, Andrea Dittadi, Anders Christensen, Ole Winther
Empirically, for the training of both continuous and discrete generative models, the proposed method yields superior variance reduction, resulting in an SNR for IWAE that increases with $K$ without relying on the reparameterization trick.
no code implementations • ICLR 2021 • Andrea Dittadi, Frederik Träuble, Francesco Locatello, Manuel Wüthrich, Vaibhav Agrawal, Ole Winther, Stefan Bauer, Bernhard Schölkopf
Learning meaningful representations that disentangle the underlying structure of the data generating process is considered to be of key importance in machine learning.
1 code implementation • 5 Aug 2020 • Valentin Liévin, Andrea Dittadi, Anders Christensen, Ole Winther
This paper introduces novel results for the score function gradient estimator of the importance weighted variational bound (IWAE).
3 code implementations • NeurIPS 2020 • Didrik Nielsen, Priyank Jaini, Emiel Hoogeboom, Ole Winther, Max Welling
Normalizing flows and variational autoencoders are powerful generative models that can represent complicated density functions.
1 code implementation • NeurIPS 2020 • Didrik Nielsen, Ole Winther
Flow models have recently made great progress at modeling ordinal discrete data such as images and audio.
no code implementations • pproximateinference AABI Symposium 2019 • Valentin Liévin, Andrea Dittadi, Lars Maaløe, Ole Winther
We introduce the Hierarchical Discrete Variational Autoencoder (HD-VAE): a hi- erarchy of variational memory layers.
no code implementations • 25 Sep 2019 • Andrea Dittadi, Ole Winther
We propose a probabilistic generative model for unsupervised learning of structured, interpretable, object-based representations of visual scenes.
2 code implementations • NeurIPS 2019 • Lars Maaløe, Marco Fraccaro, Valentin Liévin, Ole Winther
In this paper we close the performance gap by constructing VAE models that can effectively utilize a deep hierarchy of stochastic variables and model complex covariance structures.
Ranked #18 on Image Generation on ImageNet 32x32 (bpd metric)
2 code implementations • 18 Dec 2018 • Rasmus Berg Palm, Florian Laws, Ole Winther
We believe our proposed architecture can be used on many real life information extraction tasks where word classification cannot be used due to a lack of the required word-level labels.
3 code implementations • 8 Feb 2018 • Jacob Madsen, Pei Liu, Jens Kling, Jakob Birkedal Wagner, Thomas Willum Hansen, Ole Winther, Jakob Schiøtz
Recording atomic-resolution transmission electron microscopy (TEM) images is becoming increasingly routine.
Materials Science
1 code implementation • ICLR 2018 • Rasmus Berg Palm, Ulrich Paquet, Ole Winther
Humans possess an ability to abstractly reason about objects and their interactions, an ability not shared with state-of-the-art deep learning models.
no code implementations • ICLR 2018 • Dan Svenstrup, Jonas Meinertz Hansen, Ole Winther
Labeled text classification datasets are typically only available in a few select languages.
no code implementations • ICLR 2018 • Lars Maaløe, Ole Winther
There have been multiple attempts with variational auto-encoders (VAE) to learn powerful global representations of complex data using a combination of latent stochastic variables and an autoregressive model over the dimensions of the data.
6 code implementations • NeurIPS 2018 • Rasmus Berg Palm, Ulrich Paquet, Ole Winther
We achieve state of the art results on the bAbI textual question-answering dataset with the recurrent relational network, consistently solving 20/20 tasks.
Ranked #3 on Question Answering on bAbi (Mean Error Rate metric)
1 code implementation • NeurIPS 2017 • Marco Fraccaro, Simon Kamronn, Ulrich Paquet, Ole Winther
This paper takes a step towards temporal reasoning in a dynamically changing video, not in the pixel space that constitutes its frames, but in a latent space that describes the non-linear dynamics of the objects in its world.
no code implementations • NeurIPS 2017 • Dan Svenstrup, Jonas Meinertz Hansen, Ole Winther
In hash embeddings each token is represented by $k$ $d$-dimensional embeddings vectors and one $k$ dimensional weight vector.
1 code implementation • 24 Aug 2017 • Rasmus Berg Palm, Ole Winther, Florian Laws
We describe a recurrent neural network model that can capture long range context and compare it to a baseline logistic regression model corresponding to the current CloudScan production system.
1 code implementation • WS 2017 • Rasmus Berg Palm, Dirk Hovy, Florian Laws, Ole Winther
End-to-end (E2E) models, which take raw text as input and produce the desired output directly, need not depend on token-level labels.
no code implementations • 3 Apr 2017 • Lars Maaløe, Marco Fraccaro, Ole Winther
Deep generative models trained with large amounts of unlabelled data have proven to be powerful within the domain of unsupervised learning.
1 code implementation • 20 Oct 2016 • Alexander Rosenberg Johansen, Jonas Meinertz Hansen, Elias Khazen Obeid, Casper Kaae Sønderby, Ole Winther
Most existing Neural Machine Translation models use groups of characters or whole words as their unit of input and output.
no code implementations • 23 Aug 2016 • Burak Çakmak, Manfred Opper, Bernard H. Fleury, Ole Winther
Our approach extends the framework of (generalized) approximate message passing -- assumes zero-mean iid entries of the measurement matrix -- to a general class of random matrix ensembles.
1 code implementation • NeurIPS 2016 • Marco Fraccaro, Søren Kaae Sønderby, Ulrich Paquet, Ole Winther
How can we efficiently propagate uncertainty in a latent state representation with recurrent neural networks?
no code implementations • 7 Apr 2016 • Marco Fraccaro, Ulrich Paquet, Ole Winther
The estimation of normalizing constants is a fundamental step in probabilistic model comparison.
1 code implementation • 17 Feb 2016 • Lars Maaløe, Casper Kaae Sønderby, Søren Kaae Sønderby, Ole Winther
The auxiliary variables leave the generative model unchanged but make the variational distribution more expressive.
Ranked #49 on Image Classification on SVHN
5 code implementations • NeurIPS 2016 • Casper Kaae Sønderby, Tapani Raiko, Lars Maaløe, Søren Kaae Sønderby, Ole Winther
Variational Autoencoders are powerful models for unsupervised learning.
28 code implementations • 31 Dec 2015 • Anders Boesen Lindbo Larsen, Søren Kaae Sønderby, Hugo Larochelle, Ole Winther
We present an autoencoder that leverages learned representations to better measure similarities in data space.
2 code implementations • 17 Sep 2015 • Søren Kaae Sønderby, Casper Kaae Sønderby, Lars Maaløe, Ole Winther
We investigate different down-sampling factors (ratio of pixel in input and output) for the SPN and show that the RNN-SPN model is able to down-sample the input images without deteriorating performance.
no code implementations • 15 Sep 2015 • Michael Riis Andersen, Aki Vehtari, Ole Winther, Lars Kai Hansen
In this work, we address the problem of solving a series of underdetermined linear inverse problems subject to a sparsity constraint.
no code implementations • 19 Aug 2015 • Michael Riis Andersen, Ole Winther, Lars Kai Hansen
We are interested in solving the multiple measurement vector (MMV) problem for instances, where the underlying sparsity pattern exhibit spatio-temporal structure motivated by the electroencephalogram (EEG) source localization problem.
no code implementations • 6 Mar 2015 • Søren Kaae Sønderby, Casper Kaae Sønderby, Henrik Nielsen, Ole Winther
Machine learning is widely used to analyze biological sequence data.
no code implementations • 18 Jan 2015 • Lars Maaloe, Morten Arngren, Ole Winther
Applying traditional collaborative filtering to digital publishing is challenging because user data is very sparse due to the high volume of documents relative to the number of users.
no code implementations • 25 Dec 2014 • Søren Kaae Sønderby, Ole Winther
Recurrent neural networks are an generalization of the feed forward neural network that naturally handle sequential data.
no code implementations • 23 Dec 2014 • Aki Vehtari, Tommi Mononen, Ville Tolvanen, Tuomas Sivula, Ole Winther
The future predictive performance of a Bayesian model can be estimated using Bayesian cross-validation.
no code implementations • NeurIPS 2014 • Michael R. Andersen, Ole Winther, Lars K. Hansen
Sparse signal recovery addresses the problem of solving underdetermined linear inverse problems subject to a sparsity constraint.
no code implementations • 9 Sep 2014 • Ulrich Paquet, Noam Koenigstein, Ole Winther
We present a novel, scalable and Bayesian approach to modelling the occurrence of pairs of symbols (i, j) drawn from a large vocabulary.
no code implementations • 12 Jan 2013 • Manfred Opper, Ulrich Paquet, Ole Winther
A perturbative expansion is made of the exact but intractable correction, and can be applied to the model's partition function and other moments of interest.
no code implementations • NeurIPS 2009 • Ricardo Henao, Ole Winther
In this paper we present a novel approach to learn directed acyclic graphs (DAG) and factor models within the same framework while also allowing for model comparison between them.
no code implementations • NeurIPS 2008 • Manfred Opper, Ulrich Paquet, Ole Winther
We develop as series of corrections to Expectation Propagation (EP), which is one of the most popular methods for approximate probabilistic inference.