Comparison of classic statistical methods and machine learning approaches to classify readiness

MOTIVATION: Predicting physical and cognitive readiness in warfighters is critical for mission success. These predictions can be improved by identifying key biomarkers using multiple omics modalities. The MASTR-E study conducted by McKetney and colleagues is one of the most comprehensive multi-omics studies of saliva samples collected from warfighters, which also applied classic linear statistical (CLS) techniques to discover key biomarkers of readiness. Aligning with McKetney et al.'s assumptions, we operationalize readiness as a binary proxy, where pre-mission samples are labeled as 'ready' to reflect a rested, unstressed physiological baseline, while post-mission samples are labeled 'not ready' to reflect cumulative physical and cognitive load from the mission. As such, readiness here is not a direct biological or physiological construct, but an inferred state likely dominated by stress-related physiological changes. This assumption and definition is discussed further in the Introduction and Limitations sections. Here, we apply machine learning (ML) analyses to better assess generalizability, consider hidden interactions, and identify nonlinear patterns in the data. We investigated whether ML approaches could predict readiness and identify relevant biomarkers. ML models were trained on proteomics-only or metabolomics-only datasets to classify participants as ready or not ready and important model features were considered as putative biomarkers. Training and testing datasets were curated for two objectives: 1) recognize biomolecular signatures indicative of readiness within the same donor and 2) assess generalizability across warfighters by withholding donors for testing. RESULTS: Proteomics-based models achieved AUCs of 0.907±0.034 and 0.860±0.063 for Objectives 1 and 2, respectively. Metabolomics-based models achieved Objective 1 AUC of 0.994±0.007 and Objective 2 AUC of 0.993±0.010. Comparative analysis with existing literature validates the model's feature importances, but the identified putative biomarkers significantly differ from those discovered through CLS analyses, as only one ML-identified biomarker overlapping with those identified through CLS methods. We show that these ML models and identified features are more robust to noise and generalizable across participants than those identified using CLS methods. AVAILABILITY: The analysis pipelines are provided as Jupyter notebooks, including all code and documentation, and are available publicly on GitHub at {{https://github.com/netrias/ReadinessClassification}}. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.