Exploring the use of large language models for classification, clinical interpretation, and treatment recommendation in breast tumor patient records

This study aims to investigate and compare the diagnostic performance, disease interpretation reliability, and treatment recommendation capabilities of multiple advanced large language models (GPT-4o, DeepSeek-R1, and DeepSeek-V3) in breast tumor cases. It retrospectively collected comprehensive clinical records of patients with breast tumors treated at Taizhou Cancer Hospital between January and April 2024. The study evaluated the accuracy of tumor classification (benign vs. malignant), the quality of disease interpretation, and the appropriateness of treatment recommendations generated by each model. To assess the clinical interpretability and utility of the models, a comprehensive performance analysis was conducted using statistical methods. A total of 45 patients with breast tumors were included, comprising 37 benign and 8 malignant cases (43 females, 2 males). GPT-4o achieved the highest area under the curve (AUC) for tumor classification (AUC = 0.848), outperforming DeepSeek-R1 (AUC = 0.736) and DeepSeek-V3 (AUC = 0.723). However, DeLong's test indicated that the differences in AUCs among the models were not statistically significant (p > 0.05). In addition, subjective evaluations by doctors indicated that DeepSeek-R1 received the highest scores for disease interpretation (4.73 ± 0.46) and treatment recommendations (4.70 ± 0.51), with consistent ratings.