Distinguishing cancer patients’ mild and severe symptoms in radiotherapy via zero-shot and few-shot large language model-based probabilistic prompts

Monitoring symptom severity during and after radiotherapy (RT) is critical for cancer patient care, however, existing machine learning and deep learning approaches face high retraining costs and lack clinically acceptable explanations. This study aims to investigate effective prompts for LLM-based reasoning and decision-making, distinguishing cancer patients’ mild and severe status during and after radiotherapy across multiple health symptoms. To overcome these limitations, we developed a set of few-shot Large Language Model (LLM)-based predictive models operating under an anomaly detection framework (AD-LLM). Our approach leverages six novel, probabilistically designed prompts tailored for AD-LLM to identify and track patients with mild or severe symptoms, enabling accurate prediction despite limited patient-specific data. Experiments on prostate cancer patients monitoring bowel pain, depression, and sexual dysfunction problems demonstrated AD-LLM’s effectiveness in classifying symptom severity levels using these structured prompts. We evaluate the performance of different prompt strategies using AUC, AUCPR, Precision, Recall, and F1 scores, with confusion matrices and ROC/PR curves visualized to offer insights into LLM reasoning. The reasoning examples of LLMs are also provided to explain and understand their decision-making process. This method provides a promising and explainable solution for real-time symptom monitoring in RT settings, potentially improving timely intervention and patient outcomes.