no code implementations • 12 Mar 2024 • Yu Yang, Siddhartha Mishra, Jeffrey N Chiang, Baharan Mirzasoleiman
In clinical text summarization on the MIMIC-III dataset (Johnson et al., 2016), S2L again outperforms training on the full dataset using only 50% of the data.
no code implementations • 22 Dec 2023 • Benjamin Scellier, Siddhartha Mishra
Resistor networks have recently had a surge of interest as substrates for energy-efficient self-learning machines.
no code implementations • 9 Oct 2023 • Tim De Ryck, Florent Bonnet, Siddhartha Mishra, Emmanuel de Bézenac
In this paper, we investigate the behavior of gradient descent algorithms in physics-informed machine learning methods like PINNs, which minimize residuals connected to partial differential equations (PDEs).
no code implementations • 6 Jun 2023 • Francesco Di Giovanni, T. Konstantin Rusch, Michael M. Bronstein, Andreea Deac, Marc Lackenby, Siddhartha Mishra, Petar Veličković
In this paper, we provide a rigorous analysis to determine which function classes of node features can be learned by an MPNN of a given capacity.
1 code implementation • 31 May 2023 • Levi Lingsch, Mike Michelis, Emmanuel de Bezenac, Sirani M. Perera, Robert K. Katzschmann, Siddhartha Mishra
The computational efficiency of many neural operators, widely used for learning solutions of PDEs, relies on the fast Fourier transform (FFT) for performing spectral computations.
no code implementations • 20 Mar 2023 • T. Konstantin Rusch, Michael M. Bronstein, Siddhartha Mishra
Node features of graph neural networks (GNNs) tend to become more similar with the increase of the network depth.
no code implementations • 7 Feb 2023 • Léonard Equer, T. Konstantin Rusch, Siddhartha Mishra
We propose a novel multi-scale message passing neural network algorithm for learning the solutions of time-dependent PDEs.
1 code implementation • NeurIPS 2023 • Bogdan Raonić, Roberto Molinaro, Tim De Ryck, Tobias Rohner, Francesca Bartolucci, Rima Alaifari, Siddhartha Mishra, Emmanuel de Bézenac
Although very successfully used in conventional machine learning, convolution based neural network architectures -- believed to be inconsistent in function space -- have been largely ignored in the context of learning solution operators of PDEs.
no code implementations • 26 Jan 2023 • Roberto Molinaro, Yunan Yang, Björn Engquist, Siddhartha Mishra
A large class of inverse problems for PDEs are only well-defined as mappings from operators to functions.
no code implementations • 3 Oct 2022 • Samuel Lanthaler, Roberto Molinaro, Patrik Hadorn, Siddhartha Mishra
A large class of hyperbolic and advection-dominated PDEs can have solutions with discontinuities.
1 code implementation • 2 Oct 2022 • T. Konstantin Rusch, Benjamin P. Chamberlain, Michael W. Mahoney, Michael M. Bronstein, Siddhartha Mishra
We present Gradient Gating (G$^2$), a novel framework for improving the performance of Graph Neural Networks (GNNs).
Ranked #3 on Node Classification on arXiv-year
no code implementations • 18 Jul 2022 • Tim De Ryck, Siddhartha Mishra, Roberto Molinaro
Physics informed neural networks (PINNs) require regularity of solutions of the underlying PDE to guarantee accurate approximation.
no code implementations • 15 Jul 2022 • Tim De Ryck, Siddhartha Mishra
We derive rigorous bounds on the error resulting from the approximation of the solution of parametric hyperbolic scalar conservation laws with ReLU neural networks.
no code implementations • 30 May 2022 • Benjamin Scellier, Siddhartha Mishra, Yoshua Bengio, Yann Ollivier
This work establishes that a physical system can perform statistical learning without gradient computations, via an Agnostic Equilibrium Propagation (Aeqprop) procedure that combines energy minimization, homeostatic control, and nudging towards the correct response.
no code implementations • 23 May 2022 • Tim De Ryck, Siddhartha Mishra
We propose a very general framework for deriving rigorous bounds on the approximation error for physics-informed neural networks (PINNs) and operator learning architectures such as DeepONets and FNOs as well as for physics-informed operator learning.
no code implementations • 23 May 2022 • Michael Prasthofer, Tim De Ryck, Siddhartha Mishra
Existing architectures for operator learning require that the number and locations of sensors (where the input functions are evaluated) remain the same across all training and test samples, significantly restricting the range of their applicability.
7 code implementations • 16 Apr 2022 • Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Anjana Arunkumar, Arjun Ashok, Arut Selvan Dhanasekaran, Atharva Naik, David Stap, Eshaan Pathak, Giannis Karamanolakis, Haizhi Gary Lai, Ishan Purohit, Ishani Mondal, Jacob Anderson, Kirby Kuznia, Krima Doshi, Maitreya Patel, Kuntal Kumar Pal, Mehrad Moradshahi, Mihir Parmar, Mirali Purohit, Neeraj Varshney, Phani Rohitha Kaza, Pulkit Verma, Ravsehaj Singh Puri, Rushang Karia, Shailaja Keyur Sampat, Savan Doshi, Siddhartha Mishra, Sujan Reddy, Sumanta Patro, Tanay Dixit, Xudong Shen, Chitta Baral, Yejin Choi, Noah A. Smith, Hannaneh Hajishirzi, Daniel Khashabi
This large and diverse collection of tasks enables rigorous benchmarking of cross-task generalization under instructions -- training models to follow instructions on a subset of tasks and evaluating them on the remaining unseen ones.
no code implementations • 17 Mar 2022 • Tim De Ryck, Ameya D. Jagtap, Siddhartha Mishra
We prove rigorous bounds on the errors resulting from the approximation of the incompressible Navier-Stokes equations with (extended) physics informed neural networks.
1 code implementation • 4 Feb 2022 • T. Konstantin Rusch, Benjamin P. Chamberlain, James Rowbottom, Siddhartha Mishra, Michael M. Bronstein
This demonstrates that the proposed framework mitigates the oversmoothing problem.
1 code implementation • ICLR 2022 • T. Konstantin Rusch, Siddhartha Mishra, N. Benjamin Erichson, Michael W. Mahoney
We propose a novel method called Long Expressive Memory (LEM) for learning long-term sequential dependencies.
Ranked #1 on Time Series Classification on EigenWorms
no code implementations • 28 Jun 2021 • Tim De Ryck, Siddhartha Mishra
Moreover, we prove that the size of the PINNs and the number of training samples only grow polynomially with the underlying dimension, enabling PINNs to overcome the curse of dimensionality in this context.
1 code implementation • ACL 2022 • Shib Sankar Dasgupta, Michael Boratko, Siddhartha Mishra, Shriya Atmakuri, Dhruvesh Patel, Xiang Lorraine Li, Andrew McCallum
In this work, we provide a fuzzy-set interpretation of box embeddings, and learn box representations of words using a set-theoretic training objective.
no code implementations • 18 Apr 2021 • Tim De Ryck, Samuel Lanthaler, Siddhartha Mishra
We derive bounds on the error, in high-order Sobolev norms, incurred in the approximation of Sobolev-regular as well as analytic functions by neural networks with the hyperbolic tangent activation function.
1 code implementation • 9 Mar 2021 • T. Konstantin Rusch, Siddhartha Mishra
The design of recurrent neural networks (RNNs) to accurately process sequential inputs with long-time dependencies is very challenging on account of the exploding and vanishing gradient problem.
1 code implementation • ICLR 2021 • T. Konstantin Rusch, Siddhartha Mishra
Circuits of biological neurons, such as in the functional parts of the brain can be modeled as networks of coupled oscillators.
1 code implementation • 25 Sep 2020 • Siddhartha Mishra, Roberto Molinaro
We propose a novel machine learning algorithm for simulating radiative transfer.
4 code implementations • 13 Aug 2020 • Kjetil O. Lye, Siddhartha Mishra, Deep Ray, Praveen Chandrasekhar
We present a novel active learning algorithm, termed as iterative surrogate model optimization (ISMO), for robust and efficient numerical approximation of PDE constrained optimization problems.
no code implementations • 29 Jun 2020 • Siddhartha Mishra, Roberto Molinaro
Physics informed neural networks (PINNs) have recently been very successfully applied for efficiently approximating inverse problems for PDEs.
no code implementations • 29 Jun 2020 • Siddhartha Mishra, Roberto Molinaro
Physics informed neural networks (PINNs) have recently been widely used for robust and accurate approximation of PDEs.
1 code implementation • 26 May 2020 • Siddhartha Mishra, T. Konstantin Rusch
We propose a deep supervised learning algorithm based on low-discrepancy sequences as the training set.
no code implementations • 13 Dec 2019 • Tim De Ryck, Siddhartha Mishra, Deep Ray
Deep neural networks and the ENO procedure are both efficient frameworks for approximating rough functions.
1 code implementation • 20 Sep 2019 • Kjetil O. Lye, Siddhartha Mishra, Roberto Molinaro
We propose a multi-level method to increase the accuracy of machine learning algorithms for approximating observables in scientific computing, particularly those that arise in systems modeled by differential equations.
2 code implementations • 6 Jun 2019 • Ulrik Skre Fjordholm, Kjetil Lye, Siddhartha Mishra, Franziska Weber
Statistical solutions are time-parameterized probability measures on spaces of integrable functions, that have been proposed recently as a framework for global solutions and uncertainty quantification for multi-dimensional hyperbolic system of conservation laws.
Numerical Analysis Fluid Dynamics 35L65, 65M08, 65C05, 65C30
3 code implementations • 7 Mar 2019 • Kjetil O. Lye, Siddhartha Mishra, Deep Ray
Under the assumption that the underlying neural networks generalize well, we prove that the deep learning MC and QMC algorithms are guaranteed to be faster than the baseline (quasi-) Monte Carlo methods.
no code implementations • 25 Jul 2018 • Siddhartha Mishra
We propose a machine learning framework to accelerate numerical computations of time-dependent ODEs and PDEs.
1 code implementation • 30 Oct 2017 • Ulrik Skre Fjordholm, Kjetil Lye, Siddhartha Mishra
We propose efficient numerical algorithms for approximating statistical solutions of scalar conservation laws.
Numerical Analysis 35L65, 65M08, 65C05, 65C30