Heterogeneous Information Fusion for Robot-Based Automated Monitoring of Bearings in Harsh Environments via Ensemble of Classifiers with Dynamic Weighted Voting

Modern inspection mobile robots can carry multiple sensors that can provide opportunities to take advantage of the fusion of information obtained from different sensors. In real-world condition monitoring, harsh environmental conditions can significantly affect the sensor's accuracy. To address this issue in this paper, we introduced a fusion approach around information gaps to handle the portion of false information that can be captured by the employed sensors. To test our idea, we looked at various types of data, such as sounds, color images, and infrared images taken by a mobile robot inspecting a mining site to check the condition of the belt conveyor idlers. The RGB images are used to classify the rotating idlers as stuck ones (late-stage faults); on the other hand, the acoustic signals are employed to identify early-stage faults. In this work, the cyclostationary analysis approach is employed to process the captured acoustic data to visualize the bearing fault signature in the form of Cyclic Spectral Coherence. Since convolutional neural networks (CNNs) and their transfer learning (TL) forms are popular approaches for performing classification tasks, a comparison study of eight CNN-TL models was conducted to find the best models to classify different fault signatures in captured RGB images and acquired Cyclic Spectral Coherence. Finally, to combine the collected information, we suggest a method called dynamic weighted majority voting, where each model's importance is regularly adjusted for each sample based on the surface temperature of the idler taken from IR images. We demonstrate that our method of combining information from multiple classifiers can work better than using just one sensor for monitoring conditions in real-world situations.