Automated Interpretation of Fleet Incidents to Enable System Level Runtime Assurance

The lack of automated assurance case validation and automated identification of assurance violations in autonomous vehicles presents significant challenges. Insufficiencies of the validation and assumptions taken at design time necessitate mechanisms to handle violations at runtime. Active supervision of system performance is essential to prevent assurance downtime, mitigate exposed safety risks, and provide actionable feedback. This work introduces the Monitoring Assurance Indicator Validation (MASSIV) framework, which identifies key violation candidates that may lead to incidents. Identification is achieved by accessing assurance case validation fragments at the system level.Backend monitoring provides telemetry metric data from fleet operation, integrating vehicle and component levels and enabling the connection to design processes. The proposed approach leverages safety performance indicators across various system levels to validate design phase assumptions and extend multivariate monitoring at the component level. The core of this approach lies in automating the observation of safety goal threshold parameters, reflecting diverse states of safety performance through calculated graph functions across large-scale systems. A method to assess safety performance in the backend, revealing the anatomy of system insufficiencies and providing potential fleet responses, is proposed.