Exploratory association between multimodal AI-derived digital biomarkers and in-hospital mortality in adult patients with pneumonia: A proof-of-concept study

Pneumonia remains a leading cause of in-hospital mortality worldwide. Current prognostic tools such as the IDSA/ATS severity score have meaningful limitations, particularly in capturing dynamic disease progression or integrating heterogeneous biological signals. Artificial intelligence (AI) offers the opportunity to derive complementary prognostic information from routinely collected electronic health record data. This exploratory, proof-of-concept retrospective study enrolled adults (≥18 years) admitted with a primary diagnosis of acute pneumonia at Hospital Alma Máter de Antioquia (Medellín, Colombia) between January 1 and June 30, 2024. After applying pre-defined exclusion criteria, 121 patients (19 non-survivors, 15.7%) comprised the final analytic cohort. Three independent AI modules were applied: (i) a ResNet-18 deep learning model quantified lung consolidation from chest radiographs (CXRs) using Class Activation Mapping; (ii) a Spanish regular-expression natural language processing (NLP) pipeline extracted modified IDSA/ATS severity scores from clinical notes; and (iii) NeuroKit2-based quantitative heart rate variability (HRV) analysis processed electrocardiogram (ECG) signals digitised from PDF archives. Bivariate associations with all-cause in-hospital mortality were examined using logistic regression. Several features exhibited statistically significant associations with mortality under conventional thresholds: AI-quantified total lung compromise ratio (OR 8.32, 95% CI 1.23-56.29), NLP-derived IDSA/ATS severity score (OR 1.78, 95% CI 1.09-2.88), and a coherent ECG/HRV profile characterised by higher heart rate (120.1 vs. 84.4 bpm, p = 0.023), reduced RMSSD (4.1 vs. 23.5 ms, p = 0.041), reduced Poincaré SD1 (3.0 vs. 17.6 ms, p = 0.041), and T-wave amplitude reductions surviving FDR correction in multiple leads. Given the small sample size and low event count (n = 19; events-per-variable ≈ 4), all associations are preliminary and hypothesis-generating only. These proof-of-concept findings suggest that integrated multimodal AI biomarkers automatically derived from low-resource clinical data can capture a cardiopulmonary stress profile associated with pneumonia mortality, and support the design of larger prospective validation studies.