Generating fault signals for mobile robots based on multimodal knowledge and multi-channel correlation generative adversarial network

The imbalanced data limit the effectiveness of mobile robot fault diagnosis, while generating pseudo multi-sensor signals of mobile robot is an effective solution. However, existing generative methods often fail to balance the differences and correlations among channels across multi-sensor signals. To address these issues, a novel multimodal knowledge and multi-channel correlation generative adversarial network (MKMCGAN) is proposed to generate high-quality fault signals. Specifically, wavelet packet decomposition (WPD) are used to extract time-frequency features for each channel, then multi generator-discriminator pair strategy (MGDS) and a time-frequency analysis knowledge module (TFKM) are designed to bring higher similarity between the generated signal and the real signal. Subsequently, we construct a sensor data association graph and design a prior knowledge correlation module (PKM), which effectively consider the impact of inter-channel correlations on generated signals. Eventually, a novel multi-channel correlation generative adversarial network is proposed to extract time-frequency features and consider inter-channel correlations, which can generate high-quality fault signals. The effectiveness of MKMCGAN is thoroughly validated on datasets collected from a real robot fault diagnosis test bench. Experimental results indicate that MKMCGAN generates higher-quality signals compared to state-of-the-art methods.