Revisiting ADMET prediction reliability under real-world challenges in the foundation model era

As AI-driven drug discovery fully enters the era of foundation models, druggability prediction is undergoing a profound transformation from being experience-driven to "data-driven." While numerous molecular foundation models with diverse representation capabilities have emerged, there remains a lack of systematic empirical mapping regarding their frontier boundaries and intrinsic limitations when faced with real-world drug discovery challenges. To this end, this study establishes a testing benchmark encompassing four major challenges-data scarcity and Out-of-Distribution (OOD) generalization, class imbalance, "beyond Rule of 5" (bRo5) chemical space generalization, and activity cliffs-to perform large-scale benchmarking on cutting-edge architectures such as molecular foundation models (KPGT, Uni-Mol) and tabular foundation models (TabPFNv2). Our analysis reveals the current frontiers and limitations of the molecular AI field: in few-shot and OOD scenarios, the tabular foundation model TabPFNv2 demonstrates superior generalization capabilities that often outperform those of the evaluated customized molecular foundation models; to address extreme class imbalance, ensemble strategies based on undersampling provide a robust mitigation approach with a reasonable trade-off between recall and precision; and within complex bRo5 chemical spaces, the pre-trained Graph Neural Network (GNN) KPGT achieves higher predictive accuracy compared to other tested architectures in predicting cyclic peptide permeability when provided with sufficient samples. However, all model categories consistently struggle to effectively identify activity cliffs, highlighting a persistent challenge for the AI architectures evaluated in this study to capture the phenomenon where minute structural variations lead to dramatic shifts in properties. To tackle these complex real-world challenges, this study confirms that ensembling high-performance models of different modalities is an effective strategy for improving predictive robustness. Furthermore, to facilitate the low-cost screening and construction of the high-quality base models needed for such ensembles, we validated that the molecular property landscape roughness index can serve as a lightweight diagnostic tool. Taken together, these insights provide practical guidance for the evaluation and selection of contemporary druggability prediction models.Scientific contributionThis study establishes a systematic benchmark framework for druggability prediction, simulating four key real-world drug discovery scenarios-including the underexplored generalization challenge in the bRo5 chemical space-and comprehensively evaluating molecular foundation models, the tabular foundation model TabPFNv2, GNNs, Automated Machine Learning (AutoML) framework, and classical machine learning approaches, thereby overcoming limitations of prior benchmarks in task diversity and model coverage. Through large-scale train-evaluation cycles, the study reveals both scenario-specific strengths of different models and their persistent shared weaknesses. Beyond benchmarking, it demonstrates that ensembling different types of models improves predictive performance and that the molecular property landscape roughness index serves as a low-cost tool for selecting high-quality models, offering more actionable insights than previous comparison studies.