Abstract 3653: Comparison between rule-based, AI-powered and manual clinical trial matching using OncoSolver, a newly-developed clinical decision support tool

Abstract This study compared OncoSolver (OS), a novel clinical decision tool for matching oncologic patients to clinical trials (CTs), with the manual matching (MM) process at European Institute of Oncology (IEO) Phase I Unit. Twenty volunteer investigators screened for suitable CTs 100 advanced cancer patient profiles (PP) comprehensive of treatment history, pathological, genomic and clinical characteristics, modeled on publicly available precision oncology cohorts. OS used rule-based (RB) and AI modalities to match PP concurrently. AI matching used GPT4 as large language model. Results were compared for accuracy, precision, and recall, with discordant matches reviewed for eligibility assessment. Time spent and correlation between experience and performance were analyzed for MM. Analyses were performed with R version 4.3.3 and RStudio. Neuroendocrine, breast and biliary tract were the most represented cancer types (n =19, 11 and 11, respectively). Seventy-one PP had molecular alterations of interest for at least one of the 25 CTs active during the study period. PIK3CA and KRAS mutations were the most common alterations of interest (n =21 and 17). The concordance among the 3 methods was 2350/2500 (94.0%), between MM and RB 2395/2500 (95.8%), between MM and AI 2388/2500 (95.5%), and between RB and AI 2418/2500 (96.7%). The review of the discordances across the 3 methods led to the definition of 100 correct CT-PP matches, with 57 PP eligible for at least one CT. The number of false positive was 17, 26 and 46 for MM, RB and AI, while false negatives were 68, 2 and 33, respectively. Accuracy was 96.6, 98.8 and 96.7, precision 65.3, 78.2 and 57.5 and recall 32.3, 98.0 and 65.7 for MM, RB and AI. RB had the highest AUC (0.98), followed by AI (0.82) and MM (0.66). Median time for OS matching was 2 minutes per PP (range 1-4), significantly shorter than MM (4 minutes, range 1-17, p < 0.001). Recall showed a significant positive correlation with the increase of experience in MM (range 1-5 years, p =0.04). MM process is inefficient and carries the risk of missed therapeutic opportunities for cancer patients, which could negatively affect their outcomes. As a proof of concept, OS has the potential to improve patient enrolment in CTs and to reduce physician workload. Both RB and AI modalities outperformed MM in terms of recall, the most important metric in this context, and time efficiency. RB matching relies on rules derived from actual CT characteristics, requiring costly updates and maintenance to sustain performance. In contrast, AI has the advantage of continuous learning, potentially enhancing its effectiveness over time and offering more scalability. Clinical validation of OS is necessary before its integration into routine practice. To the best of our knowledge, this study is the first to compare CT matching performance of human, RB and AI approaches on the same cohort of PP. Citation Format: Edoardo Crimini, Matteo Repetto, Tommaso Fougier, Giuseppe Curigliano. Comparison between rule-based, AI-powered and manual clinical trial matching using OncoSolver, a newly-developed clinical decision support tool [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2025; Part 1 (Regular Abstracts); 2025 Apr 25-30; Chicago, IL. Philadelphia (PA): AACR; Cancer Res 2025;85(8_Suppl_1):Abstract nr 3653.