Brain Tumor Images Class-Based and Prompt-Based Detectors and Segmenter: Performance Evaluation of YOLO, SAM and Grounding DINO

Early tumor diagnosis is crucial for effective treatment and improved clinical outcomes in oncology. However, detecting and segmenting tumors in magnetic resonance imaging (MRI) poses significant challenges due to the complexity of the data and the increasing volume of medical imaging. This study analyzes machine learning-based computer vision techniques to automate these processes. We evaluate the effectiveness of generalist approaches, such as the Segment Anything Model (SAM) and Distillation of Knowledge with No Label (DINO), against class-specific methods like the You Only Look Once (YOLO) architecture. The results show that class-based models outperform prompt-based models in detection and segmentation tasks. However, the effectiveness of each model must be assessed in specific application scenarios. Class-based models remain the most reliable option for applications demanding high performance and reliability. On the other hand, in situations where precise geometric accuracy in detection or segmentation is less critical, generalist models based on prompts, like SAM and Grounding DINO, offer a viable option for rapid application development due to their adaptive nature. Hence, our findings shed light on the strengths and limitations of each approach and their clinical implications. Although prompt-based methods show promise, purpose-trained models demonstrate better tumor detection and segmentation performance.