Semantic-aware multimodal evidence integration with a lightweight large language model for rolling bearing fault diagnosis

Abstract Rolling bearings are critical components in industrial rotating machinery, and their failures may cause unexpected downtime, economic loss, and safety risks. Although deep learning and multimodal fusion have improved vibration-based fault diagnosis, two issues remain insufficiently addressed: magnitude-dominated time-frequency representations may discard phase-related impulse timing information, and heterogeneous diagnostic cues are often fused without explicit alignment in a semantically consistent representation space. This paper proposes M2-LLM-FD, a lightweight large language model-assisted framework for semantic-aware multimodal evidence integration in rolling bearing fault diagnosis. The framework converts each vibration segment into three complementary forms: time-domain patch tokens, phase-aware time-frequency tokens, and structured statistical prompts. A phase-preserving complex convolutional dimensionality reduction network is designed to process the real and imaginary components of the time-frequency representations, thereby retaining phase-sensitive information associated with fault-induced impulsive events. A structured multimodal alignment and fusion module then harmonizes temporal granularity and projects the aligned evidence into the pretrained language-model embedding space through semantic-anchor cross-modal attention. Furthermore, domain-specific prior knowledge is injected via structured statistical prompts, and the model is efficiently fine-tuned using Low-Rank Adaptation. \textcolor{red}{Extensive experiments on the CWRU, PU, and HUST datasets show that the proposed framework achieves consistent improvements in diagnostic accuracy, robustness, and cross-dataset generalization. Ablation and controlled analyses further validate the contributions of phase-aware time-frequency modeling, structured statistical prompts, and semantic-aware multimodal alignment to the overall diagnostic performance. The results indicate that combining phase-aware signal representation with large language model-compatible contextual evidence integration can effectively improve the robustness and generalization capability of intelligent bearing fault diagnosis.

Authors

Institutions

Publication Details

Journal
Measurement Science and Technology
Published
2026-11-16
DOI
https://doi.org/10.1088/1361-6501/aec28c
Primary Topic
Machine Fault Diagnosis Techniques
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Semantic-aware multimodal evidence integration with a lightweight large language model for rolling bearing fault diagnosis

Qiang Li, Zhibo Zhao, Xiancong Wu, Yinglong Yuan et al.
3 links
Measurement Science and Technology
Machine Fault Diagnosis Techniques
article

Semantic-aware multimodal evidence integration with a lightweight large language model for rolling bearing fault diagnosis

Qiang Li, Zhibo Zhao, Xiancong Wu, Yinglong Yuan, Chao Fu
article en

Abstract

Abstract Rolling bearings are critical components in industrial rotating machinery, and their failures may cause unexpected downtime, economic loss, and safety risks. Although deep learning and multimodal fusion have improved vibration-based fault diagnosis, two issues remain insufficiently addressed: magnitude-dominated time-frequency representations may discard phase-related impulse timing information, and heterogeneous diagnostic cues are often fused without explicit alignment in a semantically consistent representation space. This paper proposes M2-LLM-FD, a lightweight large language model-assisted framework for semantic-aware multimodal evidence integration in rolling bearing fault diagnosis. The framework converts each vibration segment into three complementary forms: time-domain patch tokens, phase-aware time-frequency tokens, and structured statistical prompts. A phase-preserving complex convolutional dimensionality reduction network is designed to process the real and imaginary components of the time-frequency representations, thereby retaining phase-sensitive information associated with fault-induced impulsive events. A structured multimodal alignment and fusion module then harmonizes temporal granularity and projects the aligned evidence into the pretrained language-model embedding space through semantic-anchor cross-modal attention. Furthermore, domain-specific prior knowledge is injected via structured statistical prompts, and the model is efficiently fine-tuned using Low-Rank Adaptation. \textcolor{red}{Extensive experiments on the CWRU, PU, and HUST datasets show that the proposed framework achieves consistent improvements in diagnostic accuracy, robustness, and cross-dataset generalization. Ablation and controlled analyses further validate the contributions of phase-aware time-frequency modeling, structured statistical prompts, and semantic-aware multimodal alignment to the overall diagnostic performance. The results indicate that combining phase-aware signal representation with large language model-compatible contextual evidence integration can effectively improve the robustness and generalization capability of intelligent bearing fault diagnosis.

Measurement Science and Technology
Peking University (CN), Hebei Normal University (CN)
Machine Fault Diagnosis Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon