Clinical Evidence and FDA Recalls of Artificial Intelligence–Enabled Medical Devices

Importance: Artificial Intelligence (AI)-enabled medical devices present new safety challenges related to algorithmic complexity and evolving performance. Identifying device characteristics that influence recalls would be essential to caution against premature, widespread clinical deployment and to suggest increased surveillance. Objective: To test the hypothesis that AI-enabled medical devices lacking sufficient clinical evidence at approval have a higher hazard of recall. Design, Setting, and Participants: This retrospective cohort study included AI-enabled medical devices authorized by the US Food and Drug Administration (FDA) between August 11, 1995, and August 31, 2024. Devices were assessed from FDA authorization to recall or administrative censoring up to August 31, 2024. Adverse event reports were obtained from the FDA Manufacturer and User Facility Device Experience database and by using the Coordinating Research and Evidence for Medical Devices Postmarket Surveillance (CORE-MD PMS) Tool. Reports were mapped to International Medical Device Regulators Forum (IMDRF) codes to summarize device-related issues. Exposure: Device characteristics and occurrence of device problems. Main Outcomes and Measures: The primary outcome was time from FDA authorization to recall. Time to recall was analyzed using a bayesian Weibull survival model with regularized horseshoe priors. Associations were quantified using hazard ratios (HRs) and 95% credible intervals (CrIs) derived from the bayesian model. Results: Among 903 AI-enabled medical devices, 43 (4.8%) were recalled after a median (IQR) interval of 458 (263-1092) days. Devices with missing information regarding clinical studies had a higher hazard of recall than devices with published clinical studies (HR, 1.39; 95% CrI, 0.84-3.52). Devices flagged in the CORE-MD PMS Tool (HR, 4.28; 95% CrI, 1.01-13.10) or in both databases (HR, 2.77; 95% CrI, 0.87-14.28) had higher recall hazards. IMDRF codes showed that problems related to temperature (occurred for 1 of 9 recalls; HR, 0.45; 95% CrI, 0.03-1.26) and labeling (occurred for 2 of 9 recalls; HR, 0.55; 95% CrI, 0.06-1.37) were associated with lower hazards of recall. Incorrect use of the AI-enabled device (12 of 31 devices) was also associated with a higher estimated hazard of recall (HR, 3.33; 95% CrI, 0.97-10.71). Conclusions and Relevance: In this study of 903 FDA-authorized AI-enabled medical devices, missing information on supporting clinical studies was associated with a higher chance of recall; devices with use-related problems also were associated with elevated recall hazards. These findings highlight the importance of robust clinical validation and strengthened postmarket oversight for AI-enabled devices.