Dataset curation and signal heterogeneity shape machine learning classification of innate immune transcriptomic responses

Abstract Purpose: Host-response profiling has been proposed as a strategy to support pathogen classification, particularly when direct pathogen detection is delayed. However, host-response classifiers are sensitive to methodological choices, including classifier selection and variation in immune signal strength across sampling conditions. Methods: This study uses a controlled in vitro host-response system as a testbed to evaluate how dataset curation and signal heterogeneity influence machine learning classification performance, interpretability, and generalizability. We generated transcriptomic data from cells exposed to two Gram-positive-associated agonists with distinct response magnitudes to represent within-class signal diversity relevant to host-response classification. We compared single-layer neural network (NN), random forest (RF), and generalized linear model (GLM) classifiers using overall and class-based accuracy metrics, and evaluated interpretability using class-based variable importance. Results: Across datasets, GLM and RF models outperformed NN models, indicating that simpler classifiers may be preferable for small transcriptomic host-response datasets. We further found that feature importance profiles depended strongly on the degree of biological heterogeneity represented in the training data, highlighting a tradeoff between signal-regime-specific interpretability and robustness to within-class variation. Conclusion: These findings position dataset composition as a central design consideration for host-response classification and have implications for developing more robust classifiers relevant to future infectious disease diagnostics and antimicrobial stewardship.