Utilizing ChatGPT for the responses assessment of brain tumors treated with immunotherapy based on multiple RNAO criteria: a validation study

Motivation: Multiple RANO criteria contain different treatment response assessment for brain tumors following immunotherapy (iRABT), leading to clinical management confusion. Our study is to evaluate the potential of large language models (LLM) in learning iRABT criteria within RANO criteria. Goal(s): To evaluate the performance of ChatGPT in learning iRABT criteria from multiple RANO criteria. Approach: Assess GPT 4o(GPTs) and o1 with 17 standardized queries focusing on iRABT related context within three latest RANO criteria. Results: GPT 4o demonstrated superior accuracy (p=0.0026) and clinical applicability (p=0.0006) than o1 in learning each RANO criterion. In contrast, their performances of iRABT criteria integration were unsatisfied. Impact: Our study is the first application of ChatGPT in deeper understanding of complex iRABT criteria from multiple RANO criteria, which is critical useful for improved clinical management. Better performance of ChatGPT 4o may suggest optimal selection of LLM tools.