How Good are Current\nPocket-Based 3D Generative Models?:\nThe Benchmark Set and Evaluation of Protein Pocket-Based 3D Molecular\nGenerative Models

The development of a three-dimensional (3D) molecular\ngenerative\nmodel based on protein pockets has recently attracted a lot of attention.\nThis type of model aims to achieve the simultaneous generation of\nmolecular graphs and 3D binding conformation under the constraint\nof protein binding. Various pocket-based generative models have been\nproposed; however, currently, there is a lack of systematic and objective\nevaluation metrics for these models. To address this issue, a comprehensive\nbenchmark data set, named POKMOL-3D, is proposed to evaluate protein\npocket-based 3D molecular generative models. It includes 32 protein\ntargets together with their known active compounds as a test set to\nevaluate the versatility of generation models to mimic the real-world\nscenario. Additionally, a series of two-dimensional (2D) and 3D evaluation\nmetrics with some newly created ones was integrated to assess the\nquality of generated molecular structures and their binding conformations.\nIt is expected that this work can enhance our comprehension of the\neffectiveness and weakness of current 3D generative models and stimulate\nthe discussion on challenges and useful guidance for developing the\nnext wave of molecular generative models.