A retrospective benchmark of an open-source oncology drug ranker on 2,505 MSK CHORD patients

Abstract Precision oncology drug selection currently depends on a molecular tumor board that most hospitals in the world do not have. OpenOncology is an open-source pipeline that takes a tumor’s variant file and returns a ranked, cited shortlist of drugs. We measured its agreement with the real prescribing decisions of a large academic cancer centre, using the published MSK CHORD cohort (Memorial Sloan Kettering, Nature 2024; 24,950 patients with IMPACT panel data). Of 5,190 patients who received at least one targeted therapy, 2,505 had a tumor alteration whose FDA label names that drug. On this benchmarkable subset, OpenOncology issued a prediction for 2,466 patients (98.4%). Its top three contained a drug of the correct molecular class for 2,466 of 2,466 scored patients (100.0%) and the exact drug the patient actually received for 2,094 of 2,466 (84.9%). Counting the 39 patients with no prediction as misses, the top-three class rate is 98.4% and the exact rate is 83.6%. The top-one drug matched the drug actually received in 70.6% of scored patients. A head-to-head comparison against an OncoKB-only baseline shows the full pipeline covers 315 additional patients (+ 12.6 pp), lifts top-three exact agreement by + 9.9 pp and top-three class agreement by + 12.6 pp. The scorer, the fully filtered cohort, both ranker runs, and the fusion extract are published in the project repository.