ChatGPT’s Limitations in Athlete ECG Interpretation: Evidence from a Multicenter Diagnostic Study

Background: Artificial intelligence (AI) has shown promise in the interpretation of electrocardiograms (ECGs) using signal-based deep learning models. In parallel, large language models (LLMs) have gained increasing visibility in clinical practice, including exploratory applications in ECG analysis. Whether a general-purpose LLM can meaningfully discriminate cardiovascular disease from athlete ECGs during PPS remains unknown. We aimed to evaluate the diagnostic performance of a general-purpose LLM for this task. Methods: In this multicentre diagnostic accuracy study, we evaluated a commercially available LLM (ChatGPT, version 5) in 2950 competitive athletes undergoing PPS. All athletes underwent resting 12-lead ECG, with second- and third-line investigations performed when clinically indicated. The reference outcome was confirmed cardiovascular disease after full diagnostic work-up (n = 450, 15.3%). For each ECG, the LLM generated a numeric score (0–100) representing the inferred likelihood of underlying disease using a standardized prompt and without task-specific fine-tuning. Discriminative performance was assessed using receiver operating characteristic (ROC) analysis. Misclassification patterns were analysed according to International ECG Criteria. Results: GPT-derived scores demonstrated a marked floor effect, with a median value of 0 (IQR 0–2) in both diseased and non-diseased athletes and substantial overlap between groups. The area under the ROC curve was 0.52 (95% CI 0.49–0.55), indicating performance close to random classification. At the Youden-derived threshold, 79% of athletes with confirmed disease were incorrectly classified as negative. False-negative cases were predominantly characterized by borderline ECG patterns (82%), and a substantial number of red-flag ECG abnormalities were also missed. Conclusions: In this PPS cohort, a general-purpose LLM used in a naïve configuration showed no clinically meaningful ability to discriminate between cardiovascular disease and athlete ECGs. Without task-specific training or domain adaptation, such models should not be used for diagnostic triage in athlete screening.

Authors

Institutions

Publication Details

Journal
Journal of Cardiovascular Development and Disease
Published
2026-04-29
DOI
https://doi.org/10.3390/jcdd13050191
Primary Topic
Cardiovascular Effects of Exercise
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

ChatGPT’s Limitations in Athlete ECG Interpretation: Evidence from a Multicenter Diagnostic Study

Antonia Nigro, Fiammetta Scarzella, Elena Cavarretta, Davide Mansour et al.
Journal of Cardiovascular Development and Disease
Cardiovascular Effects of Exercise
article

ChatGPT’s Limitations in Athlete ECG Interpretation: Evidence from a Multicenter Diagnostic Study

Antonia Nigro, Fiammetta Scarzella, Elena Cavarretta, Davide Mansour, Maria Rosaria Squeo, Rachele Adorisio, Alessio Pellegrino, Vittorio De Feo, Stefano Palermi, Marco Corsi, Giulia Foccardi, Nicoletta Cantarutti, Gianluigi Dorelli, Matteo Anselmino, Filippo M. Quattrini, Daniel Neunhäeuserer, G Manfredi, Emanuele Zanardo, Andrea Palermi, Alberto Livio, Silvia Giangrandi, G Di Gioia, F D A D'ascenzo, F Graziano, G M De Ferrari, Francesco Borrelli, Antonio Pelliccia, Viviana Maestrini, Elisa Lodi, Mariagrazia Modena, Mattia Cominacini, Alessandro Zorzi, Erica Brugin, Fabrizio Ricci, Marco Vecchiato, Alessandro Biffi, Andrea Saglietto, Tommaso Remo Iacovone, Riccardo Tonelli, Flavio D’Ascenzi, Sabina Gallina
article en

Abstract

Background: Artificial intelligence (AI) has shown promise in the interpretation of electrocardiograms (ECGs) using signal-based deep learning models. In parallel, large language models (LLMs) have gained increasing visibility in clinical practice, including exploratory applications in ECG analysis. Whether a general-purpose LLM can meaningfully discriminate cardiovascular disease from athlete ECGs during PPS remains unknown. We aimed to evaluate the diagnostic performance of a general-purpose LLM for this task. Methods: In this multicentre diagnostic accuracy study, we evaluated a commercially available LLM (ChatGPT, version 5) in 2950 competitive athletes undergoing PPS. All athletes underwent resting 12-lead ECG, with second- and third-line investigations performed when clinically indicated. The reference outcome was confirmed cardiovascular disease after full diagnostic work-up (n = 450, 15.3%). For each ECG, the LLM generated a numeric score (0–100) representing the inferred likelihood of underlying disease using a standardized prompt and without task-specific fine-tuning. Discriminative performance was assessed using receiver operating characteristic (ROC) analysis. Misclassification patterns were analysed according to International ECG Criteria. Results: GPT-derived scores demonstrated a marked floor effect, with a median value of 0 (IQR 0–2) in both diseased and non-diseased athletes and substantial overlap between groups. The area under the ROC curve was 0.52 (95% CI 0.49–0.55), indicating performance close to random classification. At the Youden-derived threshold, 79% of athletes with confirmed disease were incorrectly classified as negative. False-negative cases were predominantly characterized by borderline ECG patterns (82%), and a substantial number of red-flag ECG abnormalities were also missed. Conclusions: In this PPS cohort, a general-purpose LLM used in a naïve configuration showed no clinically meaningful ability to discriminate between cardiovascular disease and athlete ECGs. Without task-specific training or domain adaptation, such models should not be used for diagnostic triage in athlete screening.

Journal of Cardiovascular Development and DiseaseVol. 13(5)
University of Siena (IT), University of Verona (IT), University of Padua (IT), Azienda Ospedaliera Citta' della Salute e della Scienza di Torino (IT), National Centre for Disease Prevention and Control (IT), Saint Camillus International University of Health and Medical Sciences (IT), Azienda Ospedaliero-Universitaria di Modena (IT), Bambino Gesù Children's Hospital (IT), Institute of Sports Medicine and Science (JP), Azienda USL di Pescara (IT), Ospedale SS. Annunziata (IT), Istituti di Ricovero e Cura a Carattere Scientifico (IT), Azienda Sanitaria Locale Roma 3 (IT), Thetis (IT), Ferrari (Italy) (IT), University of Florence (IT), University of Turin (IT), Sapienza University of Rome (IT)
Reduced inequalities
Openalex Percentile: Top 9%
Cardiovascular Effects of Exercise
DOIOpen Access PDFOpenAlex
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.