Noise Ceilings for Molecular Potency Benchmarks: Measuring Label Noise from Independent ChEMBL Documents

The same compound measured against the same target in different laboratories often receives different potency values, which limits how well any model can be scored against such labels. For 17 potency tasks (target × Ki or IC50) from ChEMBL 37, we estimate this limit per task from 3,292 molecules reported in at least two source documents and express model scores as a fraction of ceiling f (0: predicting the training mean; 1: reaching the estimated noise floor). The floors range from 0.31 to 0.79 pChEMBL units, but for some tasks the replicates come from a few document pairs, and a bootstrap over document pairs then gives intervals several times wider than one over molecules. After removing molecules found in MoleculeNet and TDC, we benchmark ECFP-based gradient boosting, random forest and MLP models and frozen ChemBERTa-77M and MoLFormer-XL embeddings with a boosted-tree head under random, scaffold and temporal splits. ECFP with a random forest reaches a mean f of 0.90, 0.73 and 0.36 on the three splits (random: median 0.82, range 0.80–1.03 over the floor’s uncertainty). The frozen embeddings reach 0.69/0.66, 0.55/0.54 and 0.25/0.25 (ChemBERTa/MoLFormer) and are below ECFP with gradient boosting on every split (Wilcoxon over tasks, Holm-adjusted p < 0.05); on five tasks, a D-MPNN and a fine-tuned ChemBERTa do not close the gap on random and scaffold splits. On five task–split combinations ECFP models score below the estimated floor (f > 1), which shows that the floor is uncertain for these tasks; we discuss mis-estimation and learnable document-level offsets as explanations. We report single-test-set minimum detectable effects and a reporting protocol, and release code and data.