MMNet: Multi-Modal Network for Interpretable Hard Landing Prediction

Hard landing, characterized as a key safety concern during the aircraft landing stage, gains significant attention in the realm of aviation safety research. The event generally originates from a range of anomalies, including incorrect pilot maneuvers or abrupt changes in weather conditions. In consideration of these extraordinary circumstances, we propose the Multi-Modal Network (MMNet). Utilizing the Fourier transform, the power spectrum is capable of capturing irregular shifts observed in flight data, which are subsequently incorporated into MMNet’s analysis. Specifically, the model employs pre-trained ResNet for the extraction of feature representations from the power spectrum, and it combines these representations with the embedded input. Additionally, MMNet encompasses an internal feature extraction block that engages in representation learning via both a global feature extractor and a local feature extractor. In order to enhance the interpretability of MMNet, GradCAM is employed for visualizing its decision-making process, providing a comprehensive explanation of the rationale and effectiveness of the decisions made by MMNet. In the three experiments, we designed, and MMNet exhibits superior performance, with supplementary ablation studies further substantiating the structural effectiveness of MMNet.