A time sequence multimodal deep learning method combining acceleration and frequency domain signal for bearing fault diagnosis

The performance of traditional bearing fault diagnosis methods is substantially improved by applying deep learning methods. However, enhancement in the accuracy of prevailing unimodal methods is facing a bottleneck. In this research, a multimodal time sequential preprocessing and feature extraction method is proposed to enhance the performance of current bearing fault diagnosis methods. Generating multimodal fusion feature images is initially introduced to incorporate the acceleration feature, frequency feature, and time sequential feature simultaneously. An innovative acceleration-frequency-time triple attention (AFTTA) mechanism is sequentially developed to extract features from the multimodal fusion feature images. To validate the performance of the model proposed, classification and comparative experiments are conducted at XJTU bearing run to failure dataset. The classification experiment has demonstrated a state of art climax accuracy of $\mathbf{9 9. 9 9 3 3 1 1} \%$ in 8 -class classification. The flexibility and compatibility of the model are verified in the comparative experiment, in which the architecture integrating the AFTTA module with ResNet18 can increase the accuracy and decrease the number of epochs to convergence. This research proposes a bear fault diagnosis method with both high accuracy and exceptional flexibility and compatibility.