Provenance-Associated Confounding in Cross-Collection Comparisons of Urease Inhibitors: A Quantified Reference Scale and a Domain-Aware Alternative

Reference-library comparisons are widely used in target-focused cheminformatics to characterize the chemical space occupied by bioactive compounds. The separation obtained from such comparisons may, however, reflect differences in library provenance rather than target-related chemistry. The present study quantifies this effect using UI-Ref, a curated reference set of 1218 compounds reported in urease assays in ChEMBL, BindingDB and PubChem, for which assay- and document-level provenance was retained. Within this single target-focused collection, a random forest trained on 25 physicochemical descriptors predicted the depositing database with a receiver operating characteristic area under the curve (ROC AUC) of 0.954 (95% confidence interval, CI, 0.939–0.968) and the source publication with a median one-versus-rest AUC of 0.997, providing an empirical reference scale for provenance-associated separability. Under a conventional pairwise design, comparisons between UI-Ref and external reference libraries yielded AUCs of 0.956–0.982. After property matching and publication-grouped cross-validation, the corresponding AUCs decreased to 0.762–0.841. Because the two tasks differ in terms of class priors, sample size and validation structure, this reference scale is reported as a diagnostic covariate rather than as a decision threshold, and the residual separation was compared against null models constructed within the same design. A null using only features that describe a molecule's relative position within its source library, rather than its absolute physicochemical descriptor values, reached AUCs of 0.995–0.996 in that same task and exceeded the observed models by 0.154–0.234 (paired bootstrap; DeLong Z = −20.1 to −24.6). Models predicting an administrative provenance label within an external library achieved AUCs of 0.740–0.749, close to but below the 0.762–0.841 range obtained for collection membership. This similarity indicates that descriptor-based predictability is not specific to target-related chemistry. The residual separation exceeded the within-pair label-permutation null by 0.263–0.341, which indicates a detectable difference; however, the design cannot distinguish target-related chemistry from residual provenance structure. Following conditioning on provenance predictability, approximately 1% of compound pairs remained comparable. Matching on four physicochemical properties, moreover, left 25 to 27 of the 39 covariates visible to the models imbalanced, including thiourea and chelating motifs. Nearest-neighbor rankings changed after sample-size balancing, and the apparent overlap with approved drugs corresponded to a median nearest-neighbor Tanimoto similarity of only 0.355. Clusters in the joint chemical spaces were associated more strongly with source publication than with collection membership (adjusted mutual information 0.814 versus 0.187, respectively). No association between a structural motif and potency survived hierarchical estimation using a document-level random intercept and false discovery rate control. A revised classification cascade was therefore developed, incorporating explicit label definitions, probability calibration, a prespecified applicability domain and conformal prediction. Between 92.8% and 97.0% of the compounds in each external library fell outside the ECFP4-based applicability domain, and only 10.3% of an external screening cohort completed the cascade with an informative prediction. The measured-active-against-measured-inactive stage is reported as a narrow-scope diagnostic rather than as a general predictive stage, and every model-based stage reports empirical coverage without a conformal guarantee, because exchangeability across source publications is rejected. Provenance-associated confounding can therefore dominate conventional cross-collection comparisons and should be quantified before chemical or biological interpretations are drawn. The curated data, analysis code, trained models and domain-aware application are publicly available.