Semantic-aware multimodal evidence integration with a lightweight large language model for rolling bearing fault diagnosis
Abstract Rolling bearings are critical components in industrial rotating machinery, and their failures may cause unexpected downtime, economic loss, and safety risks. Although deep learning and multimodal fusion have improved vibration-based fault diagnosis, two issues remain insufficiently addressed: magnitude-dominated time-frequency representations may discard phase-related impulse timing information, and heterogeneous diagnostic cues are often fused without explicit alignment in a semantically consistent representation space. This paper proposes M2-LLM-FD, a lightweight large language model-assisted framework for semantic-aware multimodal evidence integration in rolling bearing fault diagnosis. The framework converts each vibration segment into three complementary forms: time-domain patch tokens, phase-aware time-frequency tokens, and structured statistical prompts. A phase-preserving complex convolutional dimensionality reduction network is designed to process the real and imaginary components of the time-frequency representations, thereby retaining phase-sensitive information associated with fault-induced impulsive events. A structured multimodal alignment and fusion module then harmonizes temporal granularity and projects the aligned evidence into the pretrained language-model embedding space through semantic-anchor cross-modal attention. Furthermore, domain-specific prior knowledge is injected via structured statistical prompts, and the model is efficiently fine-tuned using Low-Rank Adaptation. \textcolor{red}{Extensive experiments on the CWRU, PU, and HUST datasets show that the proposed framework achieves consistent improvements in diagnostic accuracy, robustness, and cross-dataset generalization. Ablation and controlled analyses further validate the contributions of phase-aware time-frequency modeling, structured statistical prompts, and semantic-aware multimodal alignment to the overall diagnostic performance. The results indicate that combining phase-aware signal representation with large language model-compatible contextual evidence integration can effectively improve the robustness and generalization capability of intelligent bearing fault diagnosis.
Authors
- Qiang Li (ORCID: https://orcid.org/0000-0001-8751-161X)
- Zhibo Zhao
- Xiancong Wu
- Yinglong Yuan (ORCID: https://orcid.org/0009-0007-0292-8493)
- Chao Fu
Institutions
- Peking University (CN)
- Hebei Normal University (CN)
Publication Details
- Journal
- Measurement Science and Technology
- Published
- 2026-11-16
- DOI
- https://doi.org/10.1088/1361-6501/aec28c
- Primary Topic
- Machine Fault Diagnosis Techniques
- Type
- article
- Field-Weighted Citation Impact
- 0.00