FireMultiFormer: A Transformer-Based Multimodal Approach for Predicting Arc Fault Fires With Fire Images and Fault Signals

The suddenness and obscurity of arc faults significantly complicate the detection and prediction of resultant fires, posing a serious threat to power systems and public safety. Accurate prediction and timely identification of fires induced by arc faults are essential. This paper proposes a novel approach for arc fault fire prediction by fusing electrical signals with fire images, instead of relying solely on electrical signal detection. A fire data collection system is developed based on an arc fault experimental setup that adheres to standard requirements. The system collects multimodal data, including diverse types of arc fault signals and fire images. By analyzing this data, the mechanisms through which arc faults initiate and influence fire evolution are elucidated. A specialized tool is developed to annotate key timestamps in multimodal data and construct a high-quality dataset for model development. To address the complex application scenarios of arc fault fires, the Fire Prediction Multimodal Fusion Transformer (FireMulti-Former) model is proposed. The model integrates fault signals and fire images to improve the accuracy of arc fault fire predictions. It uses the Patch Time Series Transformer and the Vision Transformer to extract features from current and image sequences, respectively, and employs a Fusion Transformer to fuse these multimodal features. This approach leverages data complementarity to improve the understanding and discrimination of fire states caused by arc faults. Experimental results show that the FireMultiFormer model achieves high accuracy and stability across various datasets. Case studies validate the model’s ability to recognize and learn different fire states effectively. Ablation studies elucidate the impacts of model configurations and sample parameters on performance. Compared to single-modal and alternative methods, the multimodal model significantly improves performance in fire prediction. Furthermore, the proposed model enhances the performance of conventional arc fault detection methods by integrating image data.