Evaluating Molecular Out-of-Distribution Detection Under Scaffold Shift: A Reproducible Baseline Study

Reliable assessment of molecular machine-learning systems requires evaluation under chemical distributions that differ meaningfully from the data used during model development. Molecular out-of-distribution (OOD) evaluation is therefore important for understanding how models behave when applied to chemically novel compounds.This study investigates molecular OOD detection under a strict Bemis–Murcko scaffold-disjoint setting using a ZINC15-derived molecular dataset. Molecules are represented as SMILES strings, standardized through RDKit-based canonicalization, grouped according to Bemis–Murcko scaffolds, and partitioned so that scaffolds are not shared between the in-distribution (ID) and OOD populations. The evaluation framework considers interpretable baseline approaches based on molecular fingerprints, nearest-neighbor Tanimoto similarity, and logistic regression using Morgan fingerprints.A central focus of the study is leakage-aware evaluation. During development, an initially strong similarity-based OOD signal was identified as being affected by self-similarity in the evaluation procedure. The evaluation design was subsequently revised to prevent query molecules from being used as their own similarity references. This illustrates how implementation details in molecular similarity-based OOD evaluation can substantially affect measured performance.The study is intended as a reproducible baseline and methodological analysis rather than as a new state-of-the-art OOD detection method. The results are interpreted specifically within the scaffold-disjoint experimental setting and are not intended to establish universal conclusions about molecular OOD detection. The accompanying repository provides the computational workflow and configuration used for the analysis.