Fault Detection and Diagnosis of Multi‐Joint Manipulator Based on Multi‐Information Fusion and Deep‐Learning Machine Vision

ABSTRACT The multi‐joint manipulator with vision sensors has been widely used in real applications. However, the fault detection and diagnosis accuracy are lowered and the time expense is increased for the increased number of sensors, as there are many factors that are relative with this problem. This paper is focused on the fault detection and diagnosis problem of multi‐joint manipulator, and the problem was divided into two sub‐problems. The first is that the position estimation strategy based on data fusion of visual sensor and the position sensor was designed to carry out the fault detection, and the whether the faults had happened or not were determined by the position estimation errors. The second was focused on the fault diagnosis problem, where the deep convolutional neural network (DCNN) fault diagnosis model based on time‐frequency mixed signal was constructed. The proposed DCNN uses the time and frequency domain information as its inputs and executes the classification tasks. The specific fault was determined through the output of DCNN model. The DCNN model was activated only when the first fault detection unit indicated that there was a fault, so the time expense was reduced from 5.3 to 2.6 s. The experiment based on the AUBO‐i5 manipulator was carried out to evaluate the proposed fault detection and diagnosis model, where 10 categories of data sets that represent different working conditions of manipulator were adopted. The experimental results showed that the proposed multi‐joint manipulator fault detection could improve the position estimation accuracy by 41.2%, and the fault diagnosis accuracy was improved by 20%.