From model failure to system harm: operationalizing a sociotechnical pathway for healthcare AI safety

Healthcare artificial intelligence (AI) is commonly evaluated using measures of discrimination, calibration, sensitivity, specificity, and benchmark accuracy. Although these metrics are necessary, safety ultimately depends on what occurs after an AI output enters a clinical system, whether it is noticed, trusted, verified, acted upon, and propagated through existing workflows and organizational capacities. Evidence from postmarket reports, human-AI studies, drift analyses, and equity audits demonstrates that hazards can arise at multiple points along this pathway, while remaining insufficient to quantify the incidence, attributable severity, or long-term consequences of AI-related harm. We therefore propose a five-stage sociotechnical pathway that traces risk from upstream vulnerability through AI behavior, human-workflow mediation, decision or system effect, and downstream harm. Rather than adding another catalog of governance principles, the framework follows how a specific vulnerability propagates and treats each transition as an auditable control point linked to measurable indicators, accountable actors, escalation criteria, and response actions. Thresholds should be prespecified according to the intended use and local context.