Challenges of Certifying Machine Learning Trustworthiness in High-Consequence Domains

Machine learning (ML) systems increasingly operate in high-consequence applications where trustworthiness assessment is critical, yet stakeholders often lack the information necessary to appropriately calibrate their trust in these systems. We argue that the challenge of assessing ML trustworthiness is fundamentally one of information asymmetries that exist between developers and stakeholders, and between complex models and all human actors, including their creators. These information gaps systematically impair trust development, leading to either over-trust or under-trust in ML systems. This paper advances three claims: first, that information asymmetries fundamentally impair appropriate trust development in ML systems; second, that systematic assessment and certification processes can reduce these asymmetries by creating standardized vocabularies and evaluation procedures; and third, that organizational maturity practices directly enhance trustworthiness rather than merely signaling it. We survey existing approaches to ML trustworthiness, identify critical obstacles to effective certification, and analyze methods for reducing information asymmetries through techniques such as intelligibility and provenance metadata.

Authors

Institutions

Publication Details

Journal
Data Science in Science
Published
2026-11-21
DOI
https://doi.org/10.1080/26941899.2026.2745406
Primary Topic
Explainable Artificial Intelligence (XAI)
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Challenges of Certifying Machine Learning Trustworthiness in High-Consequence Domains

Samuel Grayson, Reed M. Milewicz, Adrienne C. Kinney, Erin Acquesta et al.
Data Science in Science
Explainable Artificial Intelligence (XAI)
article

Challenges of Certifying Machine Learning Trustworthiness in High-Consequence Domains

Samuel Grayson, Reed M. Milewicz, Adrienne C. Kinney, Erin Acquesta, J. Jake Nichol, Michael Darling, Karin M. Butler, Mark A. Smith
article en

Abstract

Machine learning (ML) systems increasingly operate in high-consequence applications where trustworthiness assessment is critical, yet stakeholders often lack the information necessary to appropriately calibrate their trust in these systems. We argue that the challenge of assessing ML trustworthiness is fundamentally one of information asymmetries that exist between developers and stakeholders, and between complex models and all human actors, including their creators. These information gaps systematically impair trust development, leading to either over-trust or under-trust in ML systems. This paper advances three claims: first, that information asymmetries fundamentally impair appropriate trust development in ML systems; second, that systematic assessment and certification processes can reduce these asymmetries by creating standardized vocabularies and evaluation procedures; and third, that organizational maturity practices directly enhance trustworthiness rather than merely signaling it. We survey existing approaches to ML trustworthiness, identify critical obstacles to effective certification, and analyze methods for reducing information asymmetries through techniques such as intelligibility and provenance metadata.

Data Science in ScienceVol. 5(1)
Sandia National Laboratories (US)
Explainable Artificial Intelligence (XAI)
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon