Research on the Construction of Multimodal Large Models and Self-supervised Learning for Panoramic Perception of Power Equipment

The global information perception of power equipment is the key to supporting the efficient and stable operation of the new power system. This paper adopts the digital twin technology and constructs a new framework for panoramic perception of transformer vibration status. To address the difficulties in obtaining power defect samples, the dominance of normal samples, and the reliance on large-scale data of multi-modal large models, a power defect data enhancement method based on diffusion models is proposed. Under the premise of ensuring the rationality of the generated image structure, this method utilizes the trained multi-modal large model Qwen-VL-Max to extract the high-order semantic information of real power scene images and combines the prompt engineering technology to generate synthetic images with power defect features and high quality. Moreover, for the common problem of data sparsity in multi-modal recommendation systems, a multi-modal fusion recommendation algorithm based on collaborative self-supervised learning is proposed. This algorithm effectively enhances the representation ability of multi-modal data through the joint learning of the deep features of the data, thereby alleviating the problem of performance decline in recommendations caused by data sparsity. Experimental results show that, compared with the current mainstream multi-modal recommendation algorithms, this algorithm has significant improvements in multiple recommendation evaluation indicators.