AI-Powered Diagnostic Support in Oncology: A Comparative Case Study of GPT-4 Omni and Gemini Advanced in Breast Cancer Detection

Abstract: This study evaluates the diagnostic efficacy of GPT-4 Omni and Gemini Advanced in early breast cancer detection using publicly available medical imaging data. The aim is to assess and compare these AI models’ diagnostic accuracy against established benchmarks, contributing to the enhancement of AI-powered tools in oncology. The Cancer Imaging Archive (TCIA) and the Kaggle Breast Cancer Dataset, both open-access, were utilized to evaluate the diagnostic performance of GPT-4 Omni and Gemini Advanced. Datasets include a range of breast cancer imaging modalities (mammograms, ultrasound, MRI). The study employs a case study approach, analyzing representative image samples, comparing AI-generated diagnoses with ground truth pathology labels and expert radiologist interpretations. Key performance metrics included sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and area under the receiver operating characteristic curve (AUC). Preliminary findings indicate that GPT-4 Omni achieved an AUC of 0.88 for mammogram-based detection, exceeding the performance of Gemini Advanced (AUC=0.82) in similar tasks. Conversely, Gemini Advanced demonstrated superior performance in detecting micro-metastases in ultrasound images, achieving a sensitivity of 90%. Error analysis revealed that GPT-4 Omni's performance was affected by dense breast tissue, while Gemini Advanced occasionally misclassified benign lesions as malignant. Both GPT-4 Omni and Gemini Advanced demonstrate significant potential for augmenting breast cancer diagnostics. This study highlights the need for model-specific benchmarking and tailored integration strategies for AI tools within radiology workflows to maximize diagnostic accuracy and minimize potential errors. Future research should focus on refining algorithms to address limitations identified, enhancing explainability, and exploring personalized risk assessment. These findings have implications for improving diagnostic accuracy, potentially reducing unnecessary biopsies, and ultimately enhancing patient outcomes in breast cancer screening and detection. Keywords: GPT-4 Omni, Gemini Advanced, Breast Cancer, Medical Imaging, Artificial Intelligence, Diagnostic Accuracy, Deep Learning, Mammography, Ultrasound, MRI, Failure Mode Analysis, Open-Access Data, TCIA Dataset, Kaggle Dataset