MMOGAN: Open-Set Fault Diagnosis for Wheeled Robots Under Multimodal Data via Generative Adversarial Network

Wheeled robots are widely deployed across diverse scenarios, necessitating robust fault diagnosis to ensure operational reliability. However, their sophisticated structures, complex tasks, and dynamic working conditions lead to the emergence of new faults, constituting an open-set diagnosis problem. This challenge is further compounded by the multi-modal nature of data from heterogeneous sensors, making it difficult for existing open-set methods to efficiently address this issue due to their inability to capture the intricate interdependencies among different sensors. To solve this problem, this paper proposes a novel multi-modal open-set generative adversarial network (MMOGAN), which has the following characteristics: First, this network employs feature-level adversarial learning to simplify the generation process, and uses the discriminator for direct open-set recognition, which avoids the difficulties of synthesize complex data and limited utility of generated samples. This approach realizes adaptable and modular network architectures, enhancing its flexibility and ease of integration into various diagnostic systems. Second, an improved spatial-temporal extraction module is developed, alternating between temporal and spatial feature extraction to effectively process multi-modal data. This enhancement facilitates the adversarial learning process by providing representative and distinctive features, thereby further ensuring robust open-set fault diagnosis. Experiments are conducted on a Mecanum-wheeled robot equipped with 18 channels of heterogeneous sensors developed by our research group, and the results show that the proposed model exhibits a notable advantage over other classic and state-of-the-art comparison methods.