Comparing ChatGPT and Google Gemini in urology: Which ai model provides superior patient education on penile prosthesis?

OBJECTIVE: This study aimed to compare the performance of AI-powered chatbots ChatGPT-4 and Google Gemini in patient education on penile prostheses. Specifically, the evaluation focused on the accuracy, speed, and reproducibility of their responses to patient questions. Access to accurate and comprehensive information about penile prosthesis surgeries directly impacts patients’ decision-making processes and treatment outcomes. Therefore, examining the effectiveness of AI-powered platforms in this domain is of significant importance. MATERIAL and METHODS: Fifty questions were sourced from the “People also ask” section of Google search results. These questions were separately submitted to ChatGPT-4 and Google Gemini. The responses were independently evaluated by two experienced urologists using the Global Quality Score (GQS). Misleading information was classified as misinformation. Response times and reproducibility rates were statistically analyzed, with a significance level set at p <0.05. RESULTS: ChatGPT-4 demonstrated a higher overall GQS average compared to Google Gemini (4.9±0.31 vs. 3.45±0.94, p <0.001) and provided faster response times (12.3±2.1 seconds vs. 18.7±3.4 seconds, p <0.001). No statistically significant difference was observed in reproducibility rates between the two platforms (ChatGPT: 94%, Google Gemini: 90%, p=0.20). CONCLUSION: ChatGPT-4 outperformed Google Gemini by providing both faster and more accurate responses. These findings highlight the potential of AI-powered chatbots in patient education. However, the necessity of human oversight to ensure the accuracy of the information provided by these platforms should not be overlooked. Keywords: artificial intelligence, chatbot, patient education, penile prosthesis