Artificial Intelligence Led Telephone Triage of Patients With Suspected Head and Neck Cancer, a Feasibility Study

Head and neck cancer (HNC) is a collective term used to describe a group of cancers that arise from tissue in the head and neck region. HNC patients are diagnosed via emergency presentation, routine referral, or via the nationally approved ‘urgent suspected cancer (USC)’ (formerly ‘two-week wait’), pathways having presented with one or more of a set of ‘red flag’ signs or symptoms to their General Practitioner (GP). A validated HNC risk-calculator, the HaNC-RC-v2, which combines patient demographics, risk factors, and a set of 12 ‘red-flag’ symptoms has been developed to differentiate patients into having a high (≥ 7%) or low (< 7%) probability of having cancer [1, 2]. In 2019/2020, 227 665 patients were referred in England with suspected HNC of which 6466 were subsequently diagnosed with cancer [3]. This represents a conversion rate of 2.8%, meaning the overwhelming majority (97%) of patients referred on an urgent suspected head and neck cancer (USHNC) pathway do not have cancer. Once referred on an USHNC pathway, review in secondary care is traditionally done face-to-face by a HNC surgeon who undertakes a history and a physical examination which may include direct examination of the mucosal surfaces of the head and neck region with flexible nasendoscopy (FNE). In 2020, the global coronavirus disease pandemic (COVID-19) saw a dramatic re-rationalisation of healthcare resources, with a shift to non-contact interactions necessitated to reduce disease spread, protect population health and to prevent overwhelming hospital systems. Ear Nose and Throat (ENT) and HNC specialists were particularly vulnerable due to their interactions with the reservoirs of the virus (nasopharynx), which saw these doctors worldwide unduly affected by nosocomial infection [4]. As a result, the national governing bodies of UK HNC surgeons, advised that patients with USHNC should be assessed by telephone using the (HaNC-RC-v.2) to help risk-stratify who should be seen face-to-face [1]. Despite the recent abatement of the COVID-19 pandemic, telephone consultations have remained across a wide range of healthcare settings, seen as a solution helping to address the ongoing challenge of increasing referrals to secondary care, with limited resources to meet this referral demand. External solutions, such as automation, are increasingly being reviewed as a means to help address this ongoing capacity-demand mismatch. Ufonia, a digital health company, has created ‘Dora’, an AI-driven clinical assistant which can conduct a natural-language telephone conversation with patients [5]. Dora is a UKCA Class 1 approved medical device. In 2022, Ufonia and St George's were awarded a UK Research and Innovation Small Business Research Initiative (SBRI) healthcare grant to develop and apply this technology to HNC triage. The HNC triage conversation was based on the HaNC-RCv.2 and was co-created with patients from the ‘Heads2Gether’ HNC charity. The HNC automated conversation was based on the HaNC-RCv2. Patient and public involvement (PPI) activities involved two round-table discussions and individual technology trials with HNC patients from the ‘Heads2Gether’ HNC Charity and Ufonia conversation engineers. An external PPI expert was commissioned to oversee all PPI activities. The PPI lead contacted the ‘Heads2Gether’ charity with an explanation of the project which was then distributed to its members. Patients who volunteered to be involved were informed of the scope of the project via telephone and then in written briefing documents, they were then verbally consented by the PPI expert prior to participation. All patients were remunerated for their time according to NHS England expenses and time rates for patient-participation. The final conversation was approved by the Trust's HNC department and regional HNC multi-disciplinary team Lead. Patients referred into St George's with suspected HNC in the month of June 2022 who were having an initial telephone consultation had a clinician-supervised Dora call as part of their routine telephone consultation. The AI system conducted the call as a ‘natural language’ conversation, questioning was based on the HaNC-RC-v2 with the majority of symptomology questions requiring ‘yes/no’ answers, with categories offered where appropriate, for example, persistent versus intermittent symptoms. A clinician listening to the live autonomous conversation independently labelled answers for each component of the HaNC-RC-v2. The main outcome measures included number of completed calls, AI-automated system (Dora) and clinician data capture rate and agreement for each component of the HaNC-RC-v2, and final cancer diagnosis at 1 year. Patients were also asked a net-promoter score (NPS), how likely out of 10 they were to recommend the technology to a friend or family member [6]. The Oxford University Clinical Trials and Research Governance /Joint Research Office classified the preliminary patient public involvement (PPI) activities as ‘pre-research’ not requiring formal regulatory approvals or research governance (decision on 25 June 2021). For the clinical feasibility study the Medical Research Council and the NHS Health and Research Authority research decision support tool [7] was used to determine that this project was not a formal research project as the HaNC-RC-v2 was a standard tool used in telephone triage of USHNC at the trust. This was confirmed with the trust's Associate Medical Director for Research in March 2022. The project was therefore undertaken as a quality improvement project with standard trust information governance approvals, including clinical safety and data protection and sharing processes, sought and signed off in April 2022 (IG 2021-214). All patients at the beginning of their telephone consultation were verbally consented by their clinician to undertake an automated conversation as part of their consultation. The Equator Network SQUIRE 2.0 reporting guidelines, the framework for reporting new knowledge about how to improve healthcare, were used for the drafting of this manuscript. Twenty-nine patients had a supervised, automated Dora call as part of their care following informed consent. Age ranged from 22 to 83 years, (mean 47 years), with 19 female and 10 male patients. 100% (n = 29) of the calls were successfully completed, and 100% (n = 29) of the symptoms were satisfactorily labelled by the supervising clinician from Dora's automated conversation. Dora was able to independently capture and label 95% of all the symptoms (range 83%–100%). Of the 406 symptom labels, agreement levels between Dora and the clinical supervisor were on average 89% (range 79%–100%) (Table 1). Sixty-six percent (n = 19) were classified as ‘high’ risk of having HNC (> 7.1%), with 34% (n = 10) classified as low risk of having HNC (< 7.0%). None of the patients were ultimately diagnosed with HNC (confirmed at 1 year). Patient acceptability was high, net promoter scores ranged from 4 to 10, with a median NPS of 8 out of 10, and a calculated NPS of 56% (Figure 1). This feasibility study has shown that the novel technology of AI-led telephone calls using automated voice assistants can be delivered to patients with USHNC with a high level of data capture (95%) and accuracy (89%) when compared to clinicians. Patients on this pathway were highly likely to recommend this novel technology to a friend or family member (NPS average 8 out of 10). Zhu et al. who reviewed patient satisfaction of HNC telephone triage during the COVID-19 pandemic found patients rating telephone triage between ‘satisfied’ and ‘very satisfied’, with an overall score of 4.29 out of 5 [8]. Our study showed similar robust patient acceptability with an average NPS of 8 out of 10. This likely reflects the fact that the technology was cognisantly co-created with HNC patients helping to ensure that the conversation was appropriate and sensitive for this population. This co-creation with patients and professionals is imperative to ensure AI technologies are robustly validated, safe and appropriate for patients. Hardman et al. have published the 2020 national prospective INTEGRATE study reviewing 4568 USHNC cases that were remotely triaged via telephone consultation using the HaNC-RCv2 as recommended by ENT UK during the COVID-19 pandemic [9]. The authors conclude that remote triage which incorporates risk stratification, as offered through the HaNC-RC-v2, may both reduce unnecessary attendances of low-risk patients and facilitate targeted investigations for higher risk patients. In addition, in the context of a progressively lower HNC conversion rate number of cancers picked up from those referred in over time, and in the absence of a national screening programme, innovative solutions such as remote triage need to be considered. A significant limitation is the small size of the study (29 patients) which reduces the power of conclusions drawn and the generalizability of our findings. This was due to the strict timings of the SBRI grant funding, with a longer time than anticipated needed to obtain necessary information governance approvals at the trust therefore limiting the study to the month of June only. In addition, as the COVID-19 pandemic abated the majority of USHNC referrals at the trust were reverting to face-to-face assessment with a smaller proportion having telephone consultations. The project was however intended as a small feasibility study to show ‘proof of concept’ and form the groundwork for more formal technology trials in the future. A significant strength therefore is that, to our knowledge, this is the first real-world use of an AI-led autonomous HNC triage conversation via the telephone. This work aligns with current priorities in the HNC community, as seen in the EVEREST-HN research programme currently underway at the Royal Marsden [10]. The project has an overarching aim of improving the USHNC pathway, speeding up HNC diagnosis whilst utilising healthcare resources more efficiently. AI promises to remove human-clinician capacity limitations and provide exponentially scalable care. In a human resource dependent healthcare system that is ever increasingly feeling the pressures of such limitations, innovative technologies including AI facilitated triage may offer a means of helping address these challenges. K.M., E.L., A.H., S.L., E.O., and N.D.P. designed the work. K.M., M.C., K.R., K.Y., and E.L. acquired and analysed data. K.M., K.Y., K.R., M.C., A.H., E.L., N.D.P., S.L., and E.O. drafted, revised and approved the manuscript. K.M., K.Y., K.R., M.C., A.H., E.L., N.D.P., S.L., and E.O. agree to be accountable for all aspects of the work. Co-authors employed by Ufonia: Katrina Mason, Ernest Lim, Aisling Higham, and Nick de Pennington. The peer review history for this article is available at https://www.webofscience.com/api/gateway/wos/peer-review/10.1111/coa.14226. The data that support the findings of this study are available from the corresponding author upon reasonable request.