Laryngeal cancer diagnosis based on improved YOLOv8 algorithm

Abstract Laryngeal cancer is the most common malignant tumor in the head and neck region. The larynx, also known as the voice box, plays a crucial role in voice production and ventilation. Enhancing the diagnosis and treatment of laryngeal cancer can significantly improve patients’ prognosis and quality of life. Artificial intelligence (AI) technology shows promise as a valuable tool for diagnosing laryngeal cancer. It not only reduces the burden on endoscopists in interpreting images but also performs screening and diagnosis efficiently and accurately. However, due to the hidden and diverse nature of laryngeal cancer lesions, achieving accuracy and efficiency in AI-based diagnosis presents poses challenges. This study introduces an improved YOLOv8 algorithm named MSEC-YOLO, specifically designed for the detection and classification tasks of laryngeal cancer in endoscopic images. A novel multiscale enhanced convolution module has been introduced to improve the model’s feature extraction capabilities for small-sized targets. Additionally, a tiny fully convolutional network architecture has been employed, reducing the number of model parameters and computational costs while maintaining or enhancing performance, which is crucial for real-time medical imaging analysis. The experiments utilized a real-world endoscopic image dataset from the hospital, and the results indicated that MSEC-YOLO outperformed the original YOLOv8 model and its multi-kernel versions across multiple evaluation metrics, especially in critical categories such as malignant tumors, polyps, and papillomas, demonstrating extremely high precision and recall rates.