no code implementations • 10 Oct 2023 • Jaganmohan Chandrasekaran, Tyler Cody, Nicola McCarthy, Erin Lanus, Laura Freeman
This report presents best practices for the Test and Evaluation (T&E) of ML-enabled software systems across its lifecycle.
no code implementations • 29 May 2022 • Andres Molina-Markham, Silvia G. Ionescu, Erin Lanus, Derek Ng, Sam Sommerer, Joseph J. Rushanan
In contrast with established practices that evaluate safety using the same evaluation dataset for all vehicles, we argue that adversarial evaluation fundamentally requires a process that seeks to defeat a specific protection.
no code implementations • 28 Jan 2022 • Tyler Cody, Erin Lanus, Daniel D. Doyle, Laura Freeman
In contrast to prior work which has focused on the use of coverage in regard to the internal of neural networks, this paper considers coverage over simple features derived from inputs and outputs.
no code implementations • 25 Jan 2021 • Erin Lanus, Ivan Hernandez, Adam Dachowicz, Laura Freeman, Melanie Grande, Andrew Lang, Jitesh H. Panchal, Anthony Patrick, Scott Welch
Test and evaluation is a necessary process for ensuring that engineered systems perform as intended under a variety of conditions, both expected and unexpected.