Fair Dataset Splitting for Protein-Ligand Complex Binding Affinity Prediction

Accurate prediction of protein-ligand binding affinities is crucial for structure-based drug discovery. While machine learning methods have achieved higher accuracy on the task, recent studies have shown that these models may be exploiting biases in popular databases like PDBbind database rather than learning the true physical interactions governing binding. Specifically, models can attain high performance by merely memorizing the general tendency of certain proteins or ligands to exhibit similar binding affinities, regardless of their binding partners. To address this issue, we proposed a new dataset splitting technique specially designed for the PDBbind database. Our method performs clustering of the proteins and ligands, then combines these results to ensure that similar proteins/ligands do not appear across the training, validation, and test dataset splits. This avoids the bias in PDBbind database where certain proteins/ligands tend to exhibit consistent binding behaviors and achieves to decrease the data leakage between the training dataset and the test dataset. Using the proposed splitting, we confirmed that machine learning models struggle to achieve high accuracy when only using the protein or ligand structure as the input in contrast to other partitioning schemes. We also benchmarked several existing binding affinity prediction methods using the proposed splitting. As results, the performance of existing machine learning models degrades significantly and the overestimation of reported accuracy was revealed. Our work provides a new way to construct challenging, less biased datasets for more rigorously evaluating protein-ligand binding affinity prediction techniques. This should drive the development of models that truly learn the underlying physical interactions rather than exploiting biases in the data.