Qwen-VL2 Model with NEFTune technique For Medical Report Generation

This study highlights the growing integration of AI into the biomedical domain, focusing on developing a medical report generation model by leveraging Alibaba’s open-source vision-language model, Qwen2-VL, for supervised learning. Qwen2-VL supports multilingual interactions, covering English, Chinese, and several other languages, and is capable of real-time image and video content analysis, showcasing advanced visual understanding and generation capabilities. Despite its versatility, Qwen2-VL’s generated results lack the professionalism and reliability necessary for specialized medical reports without proper fine-tuning. To address this, a dedicated instruction fine-tuning dataset was constructed for the model, accompanied by performance evaluation using relevant metrics. Additionally, NEFTune, a method designed to enhance noise resistance, was utilized to improve the model’s performance. Experimental comparisons with other models, such as Llava, CogVLM, and Qwen-VL, demonstrated that the enhanced Qwen2-VL model achieved superior performance on the Rouge metric, highlighting its effectiveness for medical report generation.