External Validation of the EAU Guidelines Bot for Urethral Stricture: Accuracy, Completeness, and Clarity Analysis

Background/Objectives: Recently the European Association of Urology (EAU) guidelines presented the EAU Guidelines bot to assist urologists in the reading of the guidelines; however, there is a lack of up-to-date external validation. The aim of our study is to assess the accuracy, completeness, and clarity of the guidelines bot in urethral strictures. Methods: A total of 117 questions based on the EAU urethral strictures guidelines recommendations were developed. Each question was input to the EAU guidelines bot and the response was assessed by two expert urologists to assess the accuracy, completeness, and clarity. Moreover, 10 simple clinical cases were input. A 5-point Likert scale was used as a score and, in case of discrepancies, a third urologist was queried. Accuracy, completeness and clarity were assessed per chapter and per grade of recommendation. All questions and answers were recorded in an Excel file. Results: Overall 117 questions were developed. In terms of accuracy, 111/117 (95%) were defined as accurate (scores 4–5), 4/117 (3%) presented a fair accuracy (score 3), and 2/117 (2%) were deemed not accurate. In terms of completeness, 93/117 (80%) were defined as complete (scores 4–5), 22/117 (19%) presented a fair completeness (score 3), and 2/117 (2%) were deemed not complete. Finally, in terms of clarity, 104/117 (89%) were defined as clear (scores 4–5), 13/117 (11%) presented a fair clarity (score 3), and 0/109 (0%) were deemed not clear. When comparing strong and weak recommendations, no differences were recorded. Overall the answers to simple clinical cases were in line with the guidelines with good accuracy, completeness and clarity scores. Conclusions: The EAU guidelines bot represents an accurate tool for urethral stenosis guidelines. Some fine-tuning is needed to improve readability and clarity.