A Hybrid Multimodal Neural-Network-Based Anomaly Detection Model for UAVs

Unmanned aerial vehicles (UAVs) are susceptible to malfunctions caused by environmental and human factors. Anomaly detection is a critical technique for ensuring the safe operation of UAVs. However, traditional methods that rely on unimodal sensor data and neural networks often fail to capture the comprehensive characteristics of faulty UAVs, limiting their effectiveness in complex environments.To address these limitations, this paper proposes a UAV anomaly detection model that utilizes heterogeneous sensor data and multimodal neural networks (FMDNN). Firstly, FMDNN uses an interpolation-based method for fusing multi-source heterogeneous UAV sensor information. This method aligns the sensor data while simulating the distribution of UAV sensor data, enabling the FMDNN to utilize multi-sensor information to assess the overall state of the UAVs. Secondly, The FMDNN trains two neural networks of different modalities to extract nonlinear features and key structural features from the fused data by parallel computing using MLP and CNN, respectively. Finally, the feature fusion module combines the nonlinear and key structural features to create a fusion vector. The fusion vector is processed by the fully connected layer and used to determine the UAV's state through a Softmax classifier. Experimental results demonstrate that on the ALFA, TLM and UAVGPSAttack datasets, the FMDNN model achieves higher accuracy than the single-model classifier, by about 2% to 9%, for both multi and binary classification tasks. At the same time, the FMDNN model has strong robustness under classification tasks with noisy data.