CMKR-PBDM: A Transmission Line Pin-Missing Bolts Detection Method Based on Cross-Media and Knowledge Reasoning

To address the limited single-image information-processing method, insufficient global knowledge-processing capability of the traditional model, and the lack of explainability in detection results for the task of transmission line pin-missing bolts detection, the researchers propose a pin-missing bolts detection method based on cross-media and knowledge reasoning (CMKR-PBDM). First, we construct a fitting-bolt image-text pair dataset and a bolt knowledge graph (BoltKG). Subsequently, a cross-media bolt knowledge fusion model (CBKFM) is proposed, thus generating the image's overall text description by fusing the global knowledge extracted by the FB-GPT with the local knowledge captured by YOLOv8. Finally, the study proposes a large language reasoning model based on the knowledge graph (LLRM-KG), which utilizes BoltKG to guide the big language model in performing knowledge reasoning on the CBKFM output information; thus, explainable pin-missing bolts detection results are obtained. In the experimental stage, the researchers select bolts on four types of background fittings as experimental objects. The experimental results indicate that the method not only improves the accuracy of pin-missing bolts detection, but also makes the pin-missing bolts detection results explainable.