AI passed the test, but can it make the rounds?

Headlines proclaiming that AI could pass the U.S. Medical Licensing Exam (USMLE) heralded a potential breakthrough: artificial intelligence demonstrated impressive performance on assessments designed to measure human medical expertise.1 This development ignited intense debates about the future of health care, sparking predictions that AI could entirely replace human physicians.2 While AI's performance is impressive, the assertion that AI physicians are imminent is premature and misleading. This narrative oversimplifies the complexities of medical care, misrepresents the role of physicians, and, perhaps most importantly, distracts from and risks hindering the development of AI as a synergistic tool to support the delivery of care by humans to humans. The USMLE and other licensing exams assess essential medical knowledge, representing only the first rung on the clinical competence ladder. Miller's pyramid (Figure 1) illustrates how these exams assess only the foundational levels of medical knowledge: “knows” and “knows how.” While vital, these components mark just the initial steps in a physician's education. Passing the USMLE by a medical student—or by AI—indicates a readiness to begin clinical training, not preparedness for independent practice. The journey to becoming a competent physician encompasses a broader range of skills and experiences. Medical education develops critical skills crucial to safe and effective patient care. This training transcends classroom learning, integrating real-world clinical experiences where trainees must meet national standards set by the Accreditation Council for Graduate Medical Education (ACGME). The ACGME outlines six core competencies: (1) medical knowledge, (2) patient care, (3) professionalism, (4) interpersonal and communication skills, (5) practice-based learning and improvement, and (6) systems-based practice. More than just diagnostic acumen, these underscore the multifaceted nature of medical care. A physician's typical day encompasses diverse responsibilities, ranging from diagnosing rare conditions to navigating insurance issues. Each demands a unique skill set beyond medical knowledge, requiring empathy, effective communication, ethical judgment, and a deep understanding of the health care ecosystem. While AI has shown impressive capabilities in specific medical tasks like radiologic image analysis and data processing, it currently fails to replicate the full spectrum of a physician's duties. The concept of AI physicians independently practicing medicine is likely flawed. However, if we entertain the possibility, these AI doctors should undergo training comparable to that of human physicians. Passing the USMLE alone qualifies neither a medical student nor an AI system to practice independently. For AI to become a “physician,” it must master all six ACGME competencies: it would need to complete residency. While large language models (LLMs) like GPT-4 demonstrate impressive medical knowledge, they face significant challenges in practice-based learning and improvement. Real-world patient care requires understanding complex socioeconomic factors and resource allocation issues—dynamics often absent from AI training data.3 Consider a diabetic patient struggling with medication adherence due to financial constraints. AI physicians might accurately diagnose the condition, but a skilled human physician would address underlying issues, seek affordable alternatives, and communicate empathetically to encourage better self-care. To truly replicate a human physician's role, AI systems must master these multidimensional aspects of clinical practice, including causal reasoning and application of knowledge to unique patient scenarios. These nuanced skills, essential for delivering compassionate and effective care, present significant hurdles for AI physicians in medical practice. Furthermore, an AI resident would need to learn from experience, identify knowledge gaps, and set learning goals. Even the best decision support tools do not make decisions in a vacuum. These tools cannot take direct action; instead, they provide information and suggestions, presenting risks and benefits to users they do not bear responsibility for final decisions. Whether due to medicolegal considerations or due to the nuanced nature of presenting those risks to patients, ultimately, the onus resides with human professionals. As a result, AI systems learn from choices made by human operators, rather than through the independent learning expected of high-quality physicians. Interpersonal and communication skills, along with the professionalism they represent, pose the most significant hurdles for AI physicians. While LLMs can generate text and share information, they cannot interpret nonverbal cues or understand context beyond the provided information. An AI resident would need to navigate team dynamics and demonstrate qualities such as compassion and respect for patient autonomy—traits that are inherently human. Even with access to such data and the privileged position of a medical team member, would that be sufficient for effectively communicating and exhibiting professional ideals? Likely not. Interactions with AI systems fundamentally differ from human interactions, disturbing the trust central to the physician–patient relationship. The concept of an AI “residency” might advance the technology within clinical spaces, but this approach fundamentally misses the mark. We should not aim to create AI physicians that can replace human physicians. Instead, we should focus on developing tools that augment human capabilities. Patients consistently prefer human interactions in their healthcare experiences. The recent adoption of telemedicine illustrates this preference: contrary to expectations, most patients favor it as a supplement to in-person care rather than a replacement. So even if AI were to excel at all ACGME-outlined competencies, it would likely fail to address the full spectrum of patients’ needs. Despite this preference by patients, physicians spend only about 23% of their time on direct clinical care and interpersonal communication.4 Increasingly, clerical duties consume most of their time and contribute to burnout. Recent studies reveal that over 40% of physicians report feeling burned out, largely due to administrative burdens that detract from the patient care they were trained to deliver.5 Instead of aiming to create AI physicians, we should focus on developing tools that complement human clinical judgment. AI's true potential lies in addressing the circumstances that contribute to physician burnout and health care delivery inefficiencies. AI could excel at tasks requiring rapid processing of vast data, such as reviewing medical histories or screening medical images. It could also code and submit insurance claims, schedule appointments, or generate initial drafts of clinical notes. By automating routine tasks, AI could free health care from the burden of clerical work, empowering physicians to focus on aspects of care requiring uniquely human qualities and allowing them to spend more time on direct patient interaction and complex reasoning—the “art” of medicine. The integration of AI into health care isn't about choosing between human and artificial intelligence but finding ways to complement each other, creating a system that's not only more efficient but also more humane. However, like other disruptive technologies, this transition won't be simple. Its success hinges on trust, access to accurate information, and the preferences of individual patients and providers, necessitating dialogue between health care professionals, AI researchers, ethicists, and policymakers to navigate the challenges and opportunities ahead. Given this constellation of new skills necessary, AI evaluation and use may well become its own core training competency for health care professionals. Medical education might evolve to include training on effective AI system use, developing skills to critically evaluate AI-generated recommendations, and learning to integrate AI insights into patient care. This could lead to new approaches to health care delivery, where AI manages routine administrative tasks and data processing and physicians reconnect with the aspects of medicine that initially drew them to the profession. Of course, a thoughtful approach to this technology could elevate the standard of care, tackle longstanding challenges, and perhaps even redefine medical practice, but only if it stops pantomiming physicians and begins to unburden them. The result could be a healthcare system that's more insightful, efficient, and, paradoxically, more human than ever before. The authors declare no conflicts of interest.