Multimodal Pre-trained Model for Enhanced Transformer Health Assessment and Automated Fault Diagnosis

Accurate transformer health assessment is essential for power system reliability and cost-effective maintenance. Traditional methods relying on single-modality data are limited in accuracy and scope. This paper introduces a multimodal pretrained model that integrates oil chromatographic data, oil temperature data, and core ground current data to generate coherent fault analysis text. Using ResNet-based encoders and a Projector, the model aligns features from different modalities into a unified textual feature space, processed by the LLAMA3-8b large language model. The training process includes feature alignment pre-training and end-to-end fine-tuning to minimize generation loss. Experimental results show that our model outperforms traditional single-modality and previous multimodal approaches in accuracy and relevance of fault diagnosis. This system improves fault detection precision, offers comprehensive transformer health insights, and enables automated diagnostic processes, underscoring the value of multimodal data integration and advanced text generation techniques.