Performance of multimodal large language models in the Japanese surgical specialist examination

Claude 3.5 Sonnet outperformed the other models in most surgical subspecialties for image-based questions and was the only model to benefit significantly from image inclusion. These findings suggest that multimodal LLMs, particularly Claude 3.5 Sonnet, hold promise as diagnostic and educational support tools in surgical domains, and that variation in visual reasoning capabilities may account for model-level differences in image-based performance.