The role of artificial intelligence platforms in handling patient‐generated questions for Mohs micrographic surgery

Artificial intelligence, especially large language models (LLMs), has become more prevalent in medical education, offering accessible information.1-3 However, Lauck et al.'s study highlights an important reality: while LLMs can generate suitable patient-facing information with notable accuracy, challenges exist regarding their adequacy and comprehensibility.4 In their analysis of LLMs such as ChatGPT and Google Bard/Gemini as educational tools in dermatology, Lauck et al. provide a thorough evaluation reflecting the potential and challenges of AI in patient education, particularly in relation to Mohs micrographic surgery (MMS).4 The study reveals that LLMs like ChatGPT and Google Bard/Gemini are generally considered appropriate for patient education, with high ratings for appropriateness (92%) and accuracy (75% rated as “mostly accurate” or higher). Notably, ChatGPT received the highest mean accuracy score, confirming its effectiveness in delivering accurate content. However, only about one-third of the responses were regarded as sufficient for direct clinical use, indicating a significant limitation in the utility of LLMs for clinical practice, with responses often lacking the necessary specificity or conciseness required for effective patient communication in a medical setting.4 One of the main concerns highlighted in the study is the complexity of the responses. On average, the comprehensibility of LLM-generated content was rated at a tenth-grade reading level, higher than the ideal readability for patient-directed health content, which typically aims for a fifth-to-sixth-grade reading level.4 This complexity can prevent patient understanding and may reduce the effectiveness of these tools in enhancing patient education. Lauck et al. also compared LLM responses to those from traditional internet searches, finding that while LLMs generally performed better, they still fell short in certain aspects compared to the ideal patient education materials. The results suggest that while LLMs offer advancements over traditional search engines, they are not yet fully equipped to replace human guidance in clinical practice.4 The study highlights the importance of developing and refining AI tools to ensure they meet the necessary standards for patient education.4 As LLMs continue to evolve, their integration into clinical practice will require ongoing validation, standardization, and collaboration with developers to address these limitations effectively. Dermatologists and other healthcare professionals must remain aware of the potential and pitfalls of AI-driven information sources, ensuring that they complement rather than replace the complex, patient-centered communication provided by healthcare professionals.5 In conclusion, although LLMs like ChatGPT and Google Bard/Gemini represent significant advancements in medical education technology, Lauck et al.'s study emphasizes the need for continued refinement. Their findings advocate for a balanced approach where AI tools enhance but do not dominate the crucial human elements of patient education and care.