Time-Series and Image Based Multimodal Study for Partial Discharge Classification of Power Equipment in Industrial Sites

Phase Resolved Partial Discharge (PRPD) images visualise discharge patterns based on electrical phase, which helps detect insulation defects and prevent equipment failures. This study introduces a multimodal deep learning classification model combining PRPD image data and high-speed time-series signals (7,680 samples/sec), and compares its performance with single-modal baselines. The PRPD images were processed using a CNN, while time-series data were handled by an LSTM. Their features were integrated via a late fusion strategy. A dataset of 45,000 multimodal samples was constructed, including five types of partial discharges (normal, noise, surface, corona, void), three types of insulators, and nine power equipment conditions. Results showed that the CNN-only model achieved the best accuracy (99.94%), followed by the late-fusion multimodal model (99.93%) and the LSTM-only model (75.28%). The slight drop in the multimodal model’s performance was attributed to noise in the timeseries modality. These results highlight that multimodal models do not always outperform unimodal ones, especially when data quality is imbalanced. Future work should explore Transformer-based encoders for better time-series representation and cross-modal interaction mechanisms to fully exploit complementary information.