Artificial intelligence responses to penile fracture: assessing accuracy and clinical utility

OBJECTIVE: This study aims to assess the accuracy and clinical utility of artificial intelligence (AI) platforms in responding to questions related to penile fracture, a rare but urgent urological emergency. MATERIAL and METHODS: Twenty-five questions addressing key clinical aspects of penile fracture were submitted to four AI platforms: ChatGPT, Copilot, Gemini, and Perplexity. Two expert urologists evaluated each response across five domains –relevance, clarity, structure, utility, and factual accuracy– using a 5-point Likert scale. Inter-rater reliability was assessed using the intraclass correlation coefficient (ICC), and statistical comparisons were made using one-way ANOVA and Tukey’s post-hoc tests. RESULTS: Copilot and ChatGPT scored highest overall, with mean scores of 4.90 and 4.89 respectively, while Perplexity scored significantly lower (4.68; p <0.001). Copilot also achieved the highest ratings in clarity and factual accuracy. Inter-rater reliability was high, and dimensional analysis confirmed the consistent superiority of Copilot and ChatGPT in clinical relevance and clarity. CONCLUSION: While AI platforms –especially Copilot and ChatGPT– show promise in generating medically relevant content about penile fracture, limitations in factual accuracy and clinical specificity remain. Caution is advised in using these tools in urgent care settings without professional oversight. Keywords: artificial intelligence, clinical accuracy, emergency medicine, large language models, penile fracture, urology