Lineage-aware evaluation of machine learning for observed cross-genus occurrence of IncC plasmids in public databases

Introduction IncC plasmids contribute to the dissemination of antimicrobial resistance across bacterial hosts. We developed an IncC-specific model to classify plasmids belonging to highly similar cross-genus pairs observed in public databases and evaluated its performance when related lineages were kept together during validation. Methods The development cohort comprised 988 PLSDB plasmids (266 label-positive), and a separately sourced RefSeq cohort contained 250 plasmids. The operational label denotes occurrence in databases rather than experimentally measured transfer. Among five standalone learners, LightGBM with 100 foldwise-selected 4-mers (LightGBM-100) was retained, with a fixed threshold of 0.306 derived via nested random-CV Youden-index selection. Results Under random five-fold cross-validation, pooled out-of-fold AUROC and average precision (AP) were 0.960 and 0.928, respectively, but fell to 0.742 and 0.498 with primary Mash-component blocking and to 0.712 and 0.412 with more permissive blocking. At the random-CV-derived threshold, blocked sensitivity was 0.259 and 0.162, respectively. In RefSeq, LightGBM-100 achieved an AUROC of 0.937 and an AP of 0.894, although 81 of 84 positives lay within Mash distance d < 0.00124 of the development cohort and the labels partly relied on pair evidence from both databases. Biological features did not reproducibly improve the 4-mer model after lineage blocking, and family- and order-crossing outcomes showed similar limitations. Discussion This study establishes an IncC-specific, lineage-aware benchmark for predicting database-observed cross-genus occurrence and demonstrates how lineage relatedness and database representation shape predictive performance. The model may help prioritize plasmids within represented sequence backgrounds for follow-up, while the integrated biological analyses provide a basis for targeted experimental investigation.