Diagnostic Concordance Across Text and Image Workflows in Nine Consumer Multimodal Artificial Intelligence Systems for Cutaneous Vascular Anomalies: A Cross-Sectional Audit
Background/Objectives: The reliability of widely accessible multimodal artificial intelligence systems for rare or complex cutaneous vascular anomalies is uncertain. We evaluated diagnostic concordance and expert-rated management outputs across clinical-summary text, clinical-summary text plus images, and image-only workflows. Methods: In this cross-sectional audit, a purposively assembled, nonconsecutive challenge set of 511 published cases was evaluated with nine consumer web systems in new-conversation first-response sessions (13,797 final score records). Outputs were classified as top-1 concordance with the source-reported diagnosis, differential-list-only inclusion, or omission. Two vascular-anomaly specialists independently rated expected management appropriateness and safety on 5-point scales. Paired comparisons used exact McNemar, Friedman, and paired Wilcoxon tests. Results: Image-only top-1 concordance ranged from 0.4% to 7.0%, with omission rates of 71.0% to 97.8%; corresponding clinical-summary-text ranges were 32.9% to 53.0% and 11.5% to 29.4%. Image-only performance was lower than both text-containing workflows for every system. Text-plus-image workflows were not consistently superior to text alone. Conclusions: These date-stamped workflow contrasts do not estimate real-world diagnostic accuracy or isolate the causal contribution of images. They support preservation of clinical context and specialist oversight.
Authors
- Guoyong Wang (ORCID: https://orcid.org/0000-0002-8224-7706)
- Xiaonan Yang (ORCID: https://orcid.org/0000-0002-3353-6982)
- Ye Zhang (ORCID: https://orcid.org/0000-0001-9215-9024)
- Yingjie Zhu (ORCID: https://orcid.org/0000-0002-0783-0897)
- Weixin Wang
- Chaonan Wang
- Hui Bi
Institutions
- Chinese Academy of Medical Sciences & Peking Union Medical College (CN)
- Peking Union Medical College Hospital (CN)
Publication Details
- Journal
- Diagnostics
- Published
- 2026-09-24
- DOI
- https://doi.org/10.3390/diagnostics16193100
- Primary Topic
- Vascular Malformations and Hemangiomas
- Type
- article
- Field-Weighted Citation Impact
- 0.00