Library size and structure, not synthetic labels, explain apparent benchmark--experiment discrepancies in antibody machine learning

Abstract Synthetic benchmarks are widely used to develop machine-learning methods for antibody--antigen binding, but the conditions under which their conclusions transfer to measured data have been examined narrowly. Typical synthetic and measured datasets differ not only in whether labels are simulated, but also in library size, library structure, alignability and measurement noise. We show that these covarying factors, not the labels, account for the apparent differences. Holding a lattice oracle fixed and rebuilding its library as 6{,}000 mutational variants of one parent, we compared it against measured affinities for four therapeutic targets and a third-party campaign of 119{,}828 measurements. Before matching, the regimes disagreed about the search advantage, the preferred representation and architecture, and learnability decomposition. After matching, three of the four disagreements disappeared and the fourth proved unmeasurable: the search advantage fell from 1{,}620-fold to single digits, 3.3-fold against 3.0-fold measured, and a protein language model overtook one-hot encoding on the matched lattice. Surviving effects include a threefold search advantage, a Spearman cost of 0.105 from measurement noise, and an 8.1-fold overstatement by random splits. Synthetic benchmarks are faithful proxies when conditions are matched; the problem lies in uncontrolled comparison.