Usability and acceptability of a symptom checker app by patients seeking urgent care

Introduction: Diagnostic errors are common in medical care and can significantly affect patient outcomes. Symptom checker apps promise to help patients understand their symptoms, possible diagnoses and actions to take, but lack evidence on accuracy, safety and usability. Methods: This study evaluated the usability and acceptability of the Ada symptom checker app for patients in urgent primary care (PC) settings, and extended it with more diverse data from an earlier emergency department (ED) study. Patients entered their symptoms into Ada before assessment by a physician. Both the PC study and the ED study compared Ada’s diagnoses to the treating physicians' diagnoses. Usability and acceptability were evaluated using a questionnaire based on the Technology Acceptance Model (TAM). Factors influencing usability were analyzed using linear and logistic regression. Quality of clinical history was also evaluated. Results: The study included 214 urgent primary care patients, and 40 ED patients. Usability scores were generally high, 73% primary care patients’ agreed or strongly agreed with the 7 TAM usability questions, ED patients scored higher at 84% (with more direct assistance). Predictors of lower usability scores were age over 50 and limited technology experience. Sex, race, ethnicity, socioeconomic status, and educational level were not significant. In the primary care study Ada’s top 3 diagnoses matched at least 1 treating physician diagnosis in 109/172(63.3%), of cases with clear final diagnoses (sensitivity), 3 independent physicians had median matching of 124/172(72.1%) on Ada summaries. Clinical history was rated complete in two thirds of cases. Conclusion: The usability and acceptance of Ada was rated highly except by people older than 50 or with lower technology experience. Extensive research is needed to validate usability of different diagnostic tools including SCs and Large Language Models (used widely for symptom checking), across diverse populations, and their effects on patient decision-making and care-seeking behavior.