Challenges of Certifying Machine Learning Trustworthiness in High-Consequence Domains
Machine learning (ML) systems increasingly operate in high-consequence applications where trustworthiness assessment is critical, yet stakeholders often lack the information necessary to appropriately calibrate their trust in these systems. We argue that the challenge of assessing ML trustworthiness is fundamentally one of information asymmetries that exist between developers and stakeholders, and between complex models and all human actors, including their creators. These information gaps systematically impair trust development, leading to either over-trust or under-trust in ML systems. This paper advances three claims: first, that information asymmetries fundamentally impair appropriate trust development in ML systems; second, that systematic assessment and certification processes can reduce these asymmetries by creating standardized vocabularies and evaluation procedures; and third, that organizational maturity practices directly enhance trustworthiness rather than merely signaling it. We survey existing approaches to ML trustworthiness, identify critical obstacles to effective certification, and analyze methods for reducing information asymmetries through techniques such as intelligibility and provenance metadata.
Authors
- Samuel Grayson (ORCID: https://orcid.org/0000-0001-5411-356X)
- Reed M. Milewicz (ORCID: https://orcid.org/0000-0002-1701-0008)
- Adrienne C. Kinney (ORCID: https://orcid.org/0000-0002-5683-7817)
- Erin Acquesta (ORCID: https://orcid.org/0000-0003-2005-7816)
- J. Jake Nichol (ORCID: https://orcid.org/0000-0001-9713-1469)
- Michael Darling
- Karin M. Butler
- Mark A. Smith
Institutions
- Sandia National Laboratories (US)
Publication Details
- Journal
- Data Science in Science
- Published
- 2026-11-21
- DOI
- https://doi.org/10.1080/26941899.2026.2745406
- Primary Topic
- Explainable Artificial Intelligence (XAI)
- Type
- article
- Field-Weighted Citation Impact
- 0.00