Trusted Artificial Intelligence in Life-Critical and High-Risk Environments

With the growing integration of artificial intelligence (AI) into autonomous decision-making, ensuring trust in these complex systems is crucial, particularly in life-critical applications where failures can be catastrophic. Existing AI-driven autonomous technology often operates under high uncertainty due to its black-box nature, demanding greater accountability, reliability, and transparency for mission success. This paper proposes a generalizable systems engineering framework for building trust in autonomous systems, demonstrated in the context of minefield traversal, a life-critical control problem. By integrating explainable statistical models into reinforcement learning (RL), this approach evaluates subsystem accuracy and uncertainty in real time, significantly enhancing reliability. Mine detection is supported by two independent, imperfect predictors, an AI model and a human evaluator, each affected differently by varying environmental conditions. Statistical methods quantify prediction reliability, while RL optimizes decisions under uncertainty. Embedding explainable statistics into RL decision-making ensures interpretable outcomes, robust risk-based monitoring, and adaptability to changing operational parameters. This approach was tested through an agent-based simulation where AI and human detection systems collaboratively navigated uncertain minefields. Results indicate improved decision transparency, AI adaptability, and real-time risk management. Explicitly designed for generalizability, this framework presents a scalable method to establish reliable autonomous systems across various safety-critical domains. Future work will refine trust metrics and explore applications in a broader context.