Research on Application of Multimodal Data Integration in Smart Grid Fault Diagnosis

With various fault types, extensive distribution, and high real-time requirements in smart grids, efficiently integrating and utilizing massive heterogeneous data has become a key problem for fault diagnosis. On the basis of the existing multimodal data fusion model, this research designs a multi-level cross-modal Transformer (ML-CMT) neural network for smart grid fault diagnosis: the improved multi-branch convolutional network and bidirectional LSTM are used to extract features from multimodal information such as sensor time series data, thermal imaging data and text fault logs. Spatiotemporal dependence and attention allocation were learned at the feature level. Then, the adaptive cross-modal alignment and multi-level attention mechanism were proposed and pertaining to the dynamic weight allocation strategy based on Transformer to realize the deep interaction and fusion of different modal features. Lastly, a multi-phase training approach was devised, and the pre-training model was formerly employed in the global feature learning phase; then, in the latter stage, a cross-distillation strategy was introduced for fine-adjustment to enhance the precision and robustness of fault location and identity. The experiments show that this model is greatly superior to the traditional multimodal fusion method in both accuracy and speed of fault diagnosis, and achieves the efficient and accurate diagnosis of smart grid faults.