no code implementations • 11 Oct 2023 • Apoorva Nitsure, Youssef Mroueh, Mattia Rigotti, Kristjan Greenewald, Brian Belgodere, Mikhail Yurochkin, Jiri Navratil, Igor Melnyk, Jerret Ross
Using this framework, we formally develop a risk-aware approach for foundation model selection given guardrails quantified by specified metrics.
no code implementations • 4 Oct 2023 • Jiri Navratil, Benjamin Elder, Matthew Arnold, Soumya Ghosh, Prasanna Sattigeri
Accurate quantification of model uncertainty has long been recognized as a fundamental requirement for trusted AI.
no code implementations • 21 Apr 2023 • Brian Belgodere, Pierre Dognin, Adam Ivankay, Igor Melnyk, Youssef Mroueh, Aleksandra Mojsilovic, Jiri Navratil, Apoorva Nitsure, Inkit Padhi, Mattia Rigotti, Jerret Ross, Yair Schiff, Radhika Vedpathak, Richard A. Young
We introduce a holistic auditing framework that comprehensively evaluates synthetic datasets and AI models.
1 code implementation • 2 Jun 2021 • Soumya Ghosh, Q. Vera Liao, Karthikeyan Natesan Ramamurthy, Jiri Navratil, Prasanna Sattigeri, Kush R. Varshney, Yunfeng Zhang
In this paper, we describe an open source Python toolkit named Uncertainty Quantification 360 (UQ360) for the uncertainty quantification of AI models.
1 code implementation • 1 Jun 2021 • Jiri Navratil, Benjamin Elder, Matthew Arnold, Soumya Ghosh, Prasanna Sattigeri
Accurate quantification of model uncertainty has long been recognized as a fundamental requirement for trusted AI.
no code implementations • 15 Dec 2020 • Benjamin Elder, Matthew Arnold, Anupama Murthi, Jiri Navratil
We address this core problem of performance prediction uncertainty with a method to compute prediction intervals for model performance.
no code implementations • 10 Jul 2020 • Begum Taskazan, Jiri Navratil, Matthew Arnold, Anupama Murthi, Ganesh Venkataraman, Benjamin Elder
Building and maintaining high-quality test sets remains a laborious and expensive task.
no code implementations • 2 Jul 2020 • Jiri Navratil, Matthew Arnold, Benjamin Elder
Generating high quality uncertainty estimates for sequential regression, particularly deep recurrent networks, remains a challenging and open problem.
no code implementations • 28 Mar 2020 • Matthew Arnold, Jeffrey Boston, Michael Desmond, Evelyn Duesterwald, Benjamin Elder, Anupama Murthi, Jiri Navratil, Darrell Reimer
Today's AI deployments often require significant human involvement and skill in the operational stages of the model lifecycle, including pre-release testing, monitoring, problem diagnosis and model improvements.
no code implementations • 23 May 2019 • Jiri Navratil, Alan King, Jesus Rios, Georgios Kollias, Ruben Torrado, Andres Codas
We develop a proxy model based on deep learning methods to accelerate the simulations of oil reservoirs--by three orders of magnitude--compared to industry-strength physics-based PDE solvers.