Evaluating the Impact of Artificial Intelligence Scribes on Clinical Workflow and Documentation Quality: A Randomized Controlled Trial

BACKGROUND: The adoption of artificial intelligence (AI) scribes has grown rapidly in recent years, aiming to improve clinical workflow, increase efficiency, and reduce administrative burden by transcribing doctor-patient interactions. These systems capture in-office conversations and generate summaries using either standard or surgeon-specific templates. However, to date, no randomized controlled trials have rigorously assessed the proposed benefits of AI scribes. This randomized controlled trial focused on assessing AI scribe efficiency, error rate, accuracy, and patients' perceptions in comparison to traditional clinical dictation-transcription methods. METHODS: A randomized controlled trial was conducted involving 304 patients in a single-surgeon adult reconstruction practice. Patients were allocated into the folowing two groups: a conventional documentation group (using dictation transcribed by the office administrator or preconstructed templates: 140 patients) and an AI scribe group (139 patients). Timing data were recorded from when the surgeon entered a consultation room to when they began with the next patient. Additional data included administrative processing times, such as transcription and faxing of medical letters. All documentation was reviewed for medical errors by the surgeon in both groups. Grammar, spelling, and wrong wording were assessed by unblinded researchers. Letter quality was independently assessed using ChatGPT, focusing on grammar, clarity, and completeness. RESULTS: Use of AI scribes resulted in a statistically significant reduction in both physician and administrative processing times (P < 0.0001). There was no significant difference in spelling or grammatical errors between groups. However, letters produced using AI scribes were rated as significantly more complete, professional, and clear. There were 19 medical errors in the AI scribe group compared to only nine errors in the conventional group. Patients were not negatively affected by using an AI scribe, and the vast majority found it helpful. CONCLUSIONS: The use of AI scribes can effectively enhance clinical efficiency and reduce administrative workload without compromising documentation quality or patient safety. Their implementation offers meaningful benefits to both physicians and office staff, allowing more focus on patient care and other critical tasks.