Enhanced descriptive analysis capabilities of large multimodal models for intelligent substation operation and maintenance

This paper explores the application of Large Multimodal Models (LMMs) for descriptive analysis tasks in intelligent substation operation and maintenance, with a focus on enhancing the quality and utility of generated descriptions. Substations require not only accurate defect detection but also comprehensive scene understanding and detailed descriptive analysis to support maintenance decision-making. We propose a Group Relative Policy Optimization (GRPO) approach to fine-tune the Qwen2.5-VL model for generating high-quality, detailed, and reasoning-supported descriptions of substation scenes. Our method incorporates specialized reward mechanisms that evaluate semantic completeness, accuracy, and practical utility of generated descriptions. Using a dataset of 1500 high-resolution substation images, we conduct extensive experiments comparing GRPO with Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO) methods. Results demonstrate that the GRPO-enhanced model significantly outperforms other approaches across all descriptive quality metrics, achieving higher scores in semantic completeness (4.32/5.0), accuracy (4.28/5.0), defect identification (0.86), feature description (0.80), level of detail (4.25/5.0), and practical utility (4.30/5.0). Through qualitative analysis of generated descriptions, we demonstrate how the enhanced model provides maintenance personnel with more comprehensive, accurate, and actionable information, significantly improving decision support capabilities for substation operation and maintenance.