Development and Temporal Evaluation of a Tiered Clinical–Ultrasound Prediction Model and Simplified Integer Score for Time-Sensitive Pathology in Women with Acute Pelvic Pain: A Retrospective Cohort Study

Background/Objectives: Clinical prediction models may support staged triage in acute pelvic pain, but simplified integer scores can lose information and destabilise point assignments. We evaluated a tiered clinical–ultrasound prediction pathway and the stability of its simplified score within an ultrasound-documented acute-pelvic-pain cohort. Methods: This single-centre retrospective cohort included 4800 encounters in 4320 women. Encounters from 2013 to 2019 formed the development cohort (n = 3360), and patient-separated encounters from 2020 to 2022 formed the temporal-evaluation cohort (n = 1440). M1 contained clinical and laboratory predictors without imaging variables, M2 added ultrasound descriptors, and M3 added cross-sectional-imaging descriptors. Multiple imputation, ridge logistic regression, patient-level cluster bootstrap, calibration, decision-curve analysis, reader reproducibility and score-stability analyses were used. Results: Urgent pathology occurred in 20.0% of development and 18.0% of temporal encounters. Temporal AUC was 0.841 for M1 and 0.845 for M2 (difference 0.004; 95% CI 0.001–0.009). Because inclusion required documented ultrasound, this difference estimates incremental information from the selected ultrasound descriptors within an already ultrasound-documented pathway and does not estimate the pathway-level benefit of performing ultrasound. The locked integer rule classified 2.9% as low risk, with 99.6% sensitivity and 3.5% specificity. Categorisation accounted for 0.057 of the total 0.060 AUC loss, and 798 distinct score cards arose in 800 bootstrap resamples. Conclusions: Continuous M2 retained good temporal discrimination, but the small M1-M2 difference does not establish whether ultrasound improves pathway-level triage or can safely alter subsequent management. The integer score had low rule-out yield and highly variable point assignments; independent evaluation should prioritise local calibration, clinical utility and score-card stability before implementation.