Multiscale Transformers With Contrastive Learning for UAV Anomaly Detection

With the widespread application of unmanned aerial vehicles (UAVs) in various fields, anomaly detection in flight data has become increasingly important. However, temporal variations and temporal correlations in flight data challenging to achieve accurate anomaly detection. To address these issues, the multi-scale Transformers with contrastive learning for UAV anomaly detection (MTCL-UAV) is proposed to achieve adaptive multi-scale modeling. The model is based on the mixture-of-experts (MoE) architecture, and each MoE block includes a router, expert networks (ENs) and an aggregator. The router performs a temporal decomposition to select optimal scales for each sample, and the aggregator fuses the outputs of the ENs into an integrated representation. To learn the temporal correlations of flight data and improve the model’s representation, a dual attention mechanism enhanced by contrastive learning (CL-DAM) is introduced, which captures not only intra- and inter-patch correlation relationships but also neighborhood relationships among patches. Experiments on a real flight dataset demonstrate that MTCL-UAV not only achieves superior performance compared to other methods, but also exhibits strong robustness. The code is available at https://github.com/SteelHu/MTCL-UAV.