Statistical testing for uncertainty of P-NET by DeepLIFT

Abstract In precision medicine, deep learning models have become essential for identifying biomarkers and predicting diagnosis and prognosis of diseases. Although these models have demonstrated considerable success in producing accurate prognostic and diagnostic predictions, their opaque decision-making processes pose challenges for meaningful biological interpretation. In response, biologically informed neural networks (BINNs), such as P-NET, have been introduced to integrate biological domain knowledge. These models commonly employ measures like SHAP and DeepLIFT to determine the importance of their nodes. However, existing models are limited because they rely on importance estimates that does not consider statistical significance of these estimates. This study proposes a novel statistical testing approach designed to address these limitations. Specifically, it introduces two permutation tests–label and gene permutation test–aimed at identifying significant genes and pathways. Our findings show that this framework can measure the statistical significance of genes and pathways. Additionally, significant pathways are biologically associated with the metastasis mechanism of cancer, thereby enhancing both the interpretability and reliability of BINN-derived insights. This approach confidence in model outputs, thereby advancing the clinical applicability of BINNs in precision medicine.