Enhanced Rolling Bearing Fault Diagnosis Using Multimodal Deep Learning and Singular Spectrum Analysis

A decision-level multimodal fusion deep learning strategy is proposed for the effective fault detection of rolling bearings based on long-term fault signals collected from multiple sensors. First, key features are extracted from the multimodal signal set using singular spectrum analysis (SSA), and these features are transformed into a composite dataset that combines short-time Fourier transform (STFT) images and time series data. Based on this, a recursive gated convolutional neural network (RGCNN) is designed to process the STFT image data, while a 1D convolutional neural network (1DCNN) is specifically optimized for training with time series data. Furthermore, decision-level multimodal feature fusion is achieved by applying a weighted average method to integrate the features from different deep learning models, aiming to obtain more comprehensive fault prediction results. The proposed method, multimodal fusion fault detection (MFFD), is validated on the Paderborn and Ottawa rolling bearing datasets, which include various typical faults. Experimental results demonstrate the effectiveness of the proposed approach. Compared to traditional single-modality deep learning models, the proposed method shows significant improvements in fault diagnosis accuracy and generalization capability.