Innovating Nephrology Point-of-Care Ultrasonography Education: Artificial Intelligence-Assisted Curriculum Integration with Human Expert Feedback Loops

Background: Point-of-care ultrasound (POCUS) education is crucial for nephrology fellowship programs to enhance timely diagnostic accuracy, procedural guidance, and patient management. Integrating a comprehensive POCUS curriculum is essential for advancing educational quality and preparing fellows for effective clinical practice in a timely manner. This quality improvement project involved experts using AI tools to develop a Nephrology POCUS curriculum. Methods: An AI-assisted curriculum development process was conducted at the Mayo Clinic Minnesota Nephrology Fellowship Program in April 2024. The process involved multiple stages of feedback and refinement using several AI models, including GPT-4.0, Claude 3.0 Opus, Gemini Advanced, and Meta AI with Llama 3. GPT-4.0 generated initial curriculum drafts, while other AI agents provided iterative feedback for improvement. Finally, blinded Nephrology POCUS experts provided insights and expertise to refine the AI-generated curriculum further. Results: The refinement process included 12 ChatGPT-4.0 revisions and a 29 communications across four AI tools, encompassing initial feedback, subsequent revisions, and final approvals from the AI tools. It yielded in a curriculum with broadened core topics, educational methods, assessment mechanisms, and integration into rotations. POCUS experts provided positive and constructive feedback, expressing satisfaction with the overall curriculum while offering suggestions for improvement. These suggestions included placing more emphasis on using POCUS for assessing fistulas and grafts, making volume assessment open-ended, structuring training to progress fellows from novice to expert, and utilizing Qpath software for mentored scans. Experts also questioned the inclusion of specific procedures and suggested pursuing certification through ASDIN. Conclusion: The AI-assisted project effectively developed a comprehensive Nephrology POCUS curriculum at the Mayo Clinic Minnesota, significantly enhancing the educational offerings of the fellowship program. Integrating expert feedback, the curriculum emphasizes core POCUS applications, educational diversity, and robust assessment techniques. This innovative approach melds expert insights with AI capabilities, promising substantial improvements in POCUS education.