In-hospital testing of NIVPredict - an AI tool for early prediction of non-invasive ventilation outcome in acute respiratory failure

Successful non-invasive ventilation (NIV) reduces ICU length of stay, the need for intubation and the risk of death. However, patients who fail NIV and require intubation have a higher risk of death. We developed NIVPredict, an easy-to-use web-based AI tool to predict NIV outcome within two hours of initiation in patients with acute respiratory failure (ARF) from diverse aetiologies and tested its useability in a hospital setting. This study included data from immunocompromised and immunocompetent patients with hypoxemic ARF due to pneumonia, sepsis or COVID-19, and hypercapnic ARF due to acute exacerbation of chronic obstructive pulmonary disease or obesity hypoventilation syndrome. The tool uses the recently proposed Tabular Prior-Data Fitted Network (TabPFN) machine learning model and was trained using a dataset of routinely collected measurements taken within one hour after NIV initiation in 665 ARF patients from the recent RENOVATE trial in Brazil. Initial external validation of the model was conducted on a dataset of 422 ARF patients from Italy, Spain, and the USA. Subsequently, the useability of a web-based tool based on the model was tested by clinicians at the University Hospitals of North Midlands NHS Trust in the UK between December 2024 and November 2025, who applied it to data collected from 57 eligible ARF patients. The AI tool provided accurate and robust prediction of NIV outcomes and consistently outperformed conventional clinical indices across all validation settings. In internal repeated cross-validation, external validation, and in-hospital testing, the tool achieved AUCs of 0.793, 0.772, and 0.858, vs. 0.717, 0.709, and 0.693 for the best clinical index (Updated HACOR score), and balanced accuracies of 78.9%, 74.5%, and 85.0%, vs. 68.7%, 63.7%, and 67.6% for the best clinical index (HACOR or Updated HACOR score), respectively. This study demonstrates superior predictive performance, compared to current clinical indices, of an AI-based tool for NIV outcome prediction on a cohort of patients with overt-acute and acute-on-chronic respiratory failure. Clinical useability of the tool was confirmed via testing by clinicians in a hospital setting, motivating its future evaluation in prospective multi-centre studies.