When Clinical AI and Learner Reasoning Conflict: An Emerging Educational Blind Spot and a Framework for Pedagogical Response

Abstract: Increasingly, artificial intelligence (AI)-enabled clinical decision support has been incorporated into healthcare settings where trainees of medicine learn. A major pedagogical problem is emerging: at times, when an AI-generated recommendation is presented to learners as part of their training, there is disagreement between the clinical decision support recommendation and learner clinical judgment, reflecting the need to interpret probabilistic AI outputs within clinical reasoning. This article suggests that the frequency of this phenomenon represents a growing educational blind spot; if not addressed by educators, it could negatively affect the development of clinical reasoning among learners, especially in situations where AI outputs are perceived as objective or inherently authoritative rather than probabilistic or fallible, and therefore could potentially hinder the development of the learner’s professional identity. This article presents illustrative educational scenarios that show how learner-AI conflicts may occur in educational settings. After presenting the examples, it describes the educational risks that exist when there is no structure to supervise responses to these types of learner-AI conflicts. Finally, it suggests an approach using a 4-stage framework called SEED (Surface-Explore-Evaluate-Decide) to transform learner-AI conflict into an intentional opportunity to learn and provides specific ideas on how to use the SEED framework to create structures for teaching, assessment, and faculty development. In light of the growing presence of technology in all areas of education, it is essential for educators to be prepared to respond to these types of tensions to preserve the values of accountability, thoughtfulness, and humanness in education. Keywords: automation bias, trust calibration, epistemic authority, algorithmic transparency, clinical decision support systems, health professions education