Noise-robust bearing fault diagnosis via heterogeneous time-frequency multimodal fusion

Rolling bearing failures remain the predominant cause of unexpected breakdowns in industrial rotating machinery. Current deep learning–based diagnostic approaches exhibit limited robustness to noise, especially under complex operational conditions. To address this challenge, this paper proposes a Heterogeneous Time-frequency Multi-Scale Attention Network (HTMSA-Net) for rolling bearing fault diagnosis that jointly leverages complementary vibration and acoustic signals. The network adopts a dual-branch architecture, with each branch built upon a ConvNeXt V2-Atto backbone, to independently extract modality-specific features from heterogeneous time-frequency representations. Specifically, continuous wavelet transform scalograms are used for vibration signals and Mel-spectrograms for acoustic signals. An Attention-based Residual Fusion (ARF) module is proposed to address the feature imbalance between branches, which employs a Multi-Scale Channel Attention mechanism to adaptively reweight modality-specific features before concatenation. The model is initialized with ImageNet-pretrained weights and trained end-to-end. Experiments on the Ottawa UORED-VAFCLS and KAIST datasets demonstrate that HTMSA-Net achieves high diagnostic accuracy under clean conditions and maintains favorable robustness under additive white Gaussian noise ranging from 6 dB to − 6 dB, consistently outperforming mainstream architectures. Ablation studies further reveal that the heterogeneous time-frequency representations and the ARF fusion module both contribute substantially to the diagnostic performance, and the adaptive ARF fusion consistently outperforms static fusion strategies.