Mixed-methods evaluation of how a predictive model pilot intervention addresses patient non-attendance at outpatient services in an NHS Foundation Trust in England

Background: There is interest in using predictive models to address non-attendance of healthcare appointments without prior notification. Although several NHS hospital trusts have piloted predictive models for non-attendance, there is a lack of published evidence in clinical settings. <br/><br/>Objectives: This mixed-methods evaluation of the pilot of a predictive model intervention in outpatient services aimed to examine a) the effect of the intervention on patient non-attendance and b) staff engagement in the delivery of the intervention.<br/> <br/>Design: A mixed-methods study across two pilot phases. Quantitative data explored the use and impact of the predictive model on non-attendance. Z-tests were conducted to assess changes to non-attendance rates pre-pilot and in the two phases. Qualitative ethnographic work included 30 periods of observation and interviews with staff.<br/><br/>Setting and participants: Nine outpatient services in an NHS Foundation Trust that volunteered to pilot the predictive model intervention. Qualitative participants were NHS clerical and administrative staff delivering the intervention and service managers. <br/><br/>Intervention: An off-the-shelf predictive model, consisting of a cloud-based, random forest algorithm, produced a risk score of non-attendance by drawing on information from patients’ electronic health records. Staff in the pilot sites attempted to phone patients with a risk score to remind them of upcoming appointments. <br/> <br/>Results: Quantitative analysis showed that in phase one, there were low volumes of intervention calls made across services, but three of nine outpatient services significantly reduced their non-attendance rate. There was a lower overall call rate in phase two among the four remaining participating services. One significantly reduced its non-attendance rate from 20.4% to 19.1% (p&lt;0.05). Thematic analysis of the qualitative data suggests that many staff delivering the intervention struggled with actioning the intervention workflow, especially when it was not seen to fall within their defined roles. Delivering the intervention exposed unanticipated operational issues around access provision and communication. Differing perceptions of how the intervention should be prioritised, especially in the context of long waiting lists and staff resourcing issues, offer some context to the limits of the pilot’s success.<br/><br/>Conclusion: The predictive model intervention was positioned as a simple solution to address a complex problem; however, there were challenges inherent in deployment within a dynamic healthcare setting. The sustainability of the intervention and its impact on patient experience warrant further exploration.