A Computational Community Blind Challenge on Pan-Coronavirus Drug Discovery Data

Computational blind challenges offer critical, unbiased assessment opportunities to assess and accelerate scientific progress, as demonstrated by a breadth of breakthroughs over the last decade. We report the outcomes and key insights from an open science community blind challenge focused on computational methods in drug discovery, using lead optimization data from the AI-driven Structure-enabled Antiviral Platform (ASAP) Discovery Consortium’s pan-coronavirus antiviral discovery program, in partnership with Polaris and the OpenADMET project. This collaborative initiative invited global participants from both academia and industry to develop and apply computational methods to predict the biochemical potency and crystallographic ligand poses of small molecules against key coronavirus targets, Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) and Middle East Respiratory Syndrome Coronavirus (MERS-CoV) main protease (Mpro), as well as multiple ADMET assay endpoints, using previously undisclosed comprehensive experimental drug discovery datasets as benchmarks. By evaluating submissions across multiple tasks and compounds, we established performance leaderboards and conducted meta-analyses to assess methodological strengths, common pitfalls, and areas for improvement. This analysis provides a foundation for best practices in real-world machine learning evaluation, grounded in community-driven benchmarking. We also highlight how next-generation platforms, such as Polaris, enable rigorous challenge design, embedded evaluation frameworks, and broad community engagement. This paper reports the collective findings of the challenge, offering a high-level overview of the data, evaluation infrastructure, and top- performing strategies. We further provide context and support for the accompanying papers authored by the challenge participants in this special issue, which explore individual approaches in greater depth. Together, these contributions aim to advance reproducible, trustworthy, and high-impact computational methods in drug discovery, and to explore best practices and pitfalls in future blind challenge design and execution, including planned initiatives for the OpenADMET project.

Authors

Institutions

Publication Details

Journal
ChemRxiv
Published
2025-09-01
DOI
https://doi.org/10.26434/chemrxiv-2025-zd9mr-v3
Primary Topic
Genetics, Bioinformatics, and Biomedical Research
Type
article
Field-Weighted Citation Impact
0.00

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

A Computational Community Blind Challenge on Pan-Coronavirus Drug Discovery Data

Stéphane De Cesco, Satya Pratik Srivastava, Giovanni Tricarico, Daniel J. Cole et al.
ChemRxiv
Genetics, Bioinformatics, and Biomedical Research
article

A Computational Community Blind Challenge on Pan-Coronavirus Drug Discovery Data

Stéphane De Cesco, Satya Pratik Srivastava, Giovanni Tricarico, Daniel J. Cole, Antonia S. J. S. Mey, Wim Dehaen, Carles Corbi‐Verge, E. Jiang, Jerry Dwi Trijoyo Purnomo, Benjamin Kaminow, Valerij Talagayev, Charles W.E. Tomlinson, Johan Hofmans, Floriane Montanari, Ruel Cedeno, Azmine Toushik Wasi, Dave West, Nick Lynch, Xiaolin Pan, Iva Mayer, Irfan Alibay, Jeremy Hsu, Bruce A. Lefker, Ralph P. Robinson, Austin Tripp, Rajeev Kumar Singh, Christoph Schran, Alexandre Tkatchenko, Chuanhui Xu, Tamás Szommer, Blake Balcomb, Kate Huddleston, Galit Cohen, Peter Sjö, Joshua T. Horton, Lianjin Cai, Jessica Stacey, Hugo MacDermott-Opeskin, Julien Michel, Gunjan Baid, Marie-Pierre Dréanic, Brian Birnbaum, Haim Michael Barr, Ashok Palaniappan, Kenneth Goossens, Noa Lahav, Kevin P. Bishop, Michael Lavi, James S. Fraser, Ed Griffen, Lauren Reid, Teresa Head‐Gordon, Long-Hung Dinh Pham, F. von Delft, Ryan Lithgo, Bathmapriya Balakrishnan, Jackson W. Burns, Eelke B. Lenselink, Wojtek Treyde, Oscar Méndez‐Lucio, Kunyang Sun, Jenke Scheen, Hugo J. Bohórquez, Yaowen Gu, Patrick Maher, Vladimir Chupakhin, M. Haleem, J.C. Aschenbrenner, Taoyu Niu, Finlay Clark, Zhaoxi Sun, Alpha A. Lee, Alyssa Ehrlich, Karla Kirkegaard, Alexander Matthew Payne, W. Patrick Walters, Mallory R. Tollefson, Gerhard Wolber, Soheila Khosravi, Zhirong Liu, Asha Patnaik, D. Fearon, Max Winokan, Anthony Gitter, John D. Chodera, Sean Colby, David L. Cousins, L. Masfarre Pinto, Maria A. Castellanos, Lars L. Schaaf, Cas Wognum, Peter Marples, Rokas Elijošius, Sijie Liu, Lu Zhu, Yingze Wang, Weijun Zhou, Alexander Rich, Bing Xu, Laurent Hoffer, Yue Liu, Ashmita Bose, Ida Titus, Mounika Srilakshmi, Alec Davi, Andrew Quirke, C. J. Brown, Muhammad Danial, Yizhou Ma, Niklas Piet Doering, Alexis Dougha, Gwendolyn Fate, Shirley Duberstein, Julien St-Laurent, Honore Etsmoberg, Bryce Eakin, Jozef Fülöp, Asma Feriel Khoualdi, Ryusei Ogino, Nopsinth Vithayapalert, Steffen Wedig
article en

Abstract

Computational blind challenges offer critical, unbiased assessment opportunities to assess and accelerate scientific progress, as demonstrated by a breadth of breakthroughs over the last decade. We report the outcomes and key insights from an open science community blind challenge focused on computational methods in drug discovery, using lead optimization data from the AI-driven Structure-enabled Antiviral Platform (ASAP) Discovery Consortium’s pan-coronavirus antiviral discovery program, in partnership with Polaris and the OpenADMET project. This collaborative initiative invited global participants from both academia and industry to develop and apply computational methods to predict the biochemical potency and crystallographic ligand poses of small molecules against key coronavirus targets, Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) and Middle East Respiratory Syndrome Coronavirus (MERS-CoV) main protease (Mpro), as well as multiple ADMET assay endpoints, using previously undisclosed comprehensive experimental drug discovery datasets as benchmarks. By evaluating submissions across multiple tasks and compounds, we established performance leaderboards and conducted meta-analyses to assess methodological strengths, common pitfalls, and areas for improvement. This analysis provides a foundation for best practices in real-world machine learning evaluation, grounded in community-driven benchmarking. We also highlight how next-generation platforms, such as Polaris, enable rigorous challenge design, embedded evaluation frameworks, and broad community engagement. This paper reports the collective findings of the challenge, offering a high-level overview of the data, evaluation infrastructure, and top- performing strategies. We further provide context and support for the accompanying papers authored by the challenge participants in this special issue, which explore individual approaches in greater depth. Together, these contributions aim to advance reproducible, trustworthy, and high-impact computational methods in drug discovery, and to explore best practices and pitfalls in future blind challenge design and execution, including planned initiatives for the OpenADMET project.

ChemRxiv
Quanta Technology (United States) (US), Ontario Institute for Cancer Research (CA), University of Vienna (AT), Drugs for Neglected Diseases Initiative (CH), Shahjalal University of Science and Technology (BD), Memorial Sloan Kettering Cancer Center (US), University of Wisconsin–Madison (US), Mashhad University of Medical Sciences (IR), University of Pittsburgh (US), Shenzhen University (CN), University of Luxembourg (LU), Peking University (CN), Université Paris Cité (FR), University of Cambridge (GB), Shiv Nadar University (IN), Post University (US), Diamond Light Source (GB), University of Oxford (GB), Alpha Technologies (Canada) (CA), Thames Water (United Kingdom) (GB), Pekin Public Library (US), Novavax (United States) (US), Japan Tobacco (Japan) (JP), Vetenskap I Skolan (SE), Universidad Católica de Santa Fe (AR), Relay Therapeutics (United States) (US), Universitas Ibrahimy (ID), State Key Laboratory of Synthetic Chemistry (CN), University of Bío-Bío (CL), Recursion (United States) (US), Alpha Environmental (United States) (US), 21c Consultancy (United Kingdom) (GB), Open Molecular Software Foundation, Imperial College London (GB), Weizmann Institute of Science (IL), New York University (US), Massachusetts Institute of Technology (US), Freie Universität Berlin (DE), Newcastle University (GB), Simulations Plus (United States) (US), University of Chemistry and Technology, Prague (CZ), SASTRA University (IN), University of California, Berkeley (US), Stanford University (US), University of Edinburgh (GB)
Oral and Maxillofacial Surgery Foundation, University of California, San Francisco
Openalex Percentile: Top 80%
Genetics, Bioinformatics, and Biomedical Research
DOIOpen Access PDFOpenAlex
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.