Student-Pilot Error Prediction via Multimodal Physiological Signals and Tree-Based Models

The United States Air Force (USAF) faces significant challenges in training new pilots to meet the growing demand for skilled airmen in various essential operations. While varied in nature, these challenges are underpinned by significant resource constraints, e.g., limited trainer aircraft ability, shortage of instructor pilots, and weather-related limitations. Recent USAF initiatives leverage simulator flights in training programs to ameliorate such issues; however, given that the USAF also faces a lack of available simulator instructors, resource limitations are not entirely circumvented. Therefore, this study aims to alleviate the strain on instructors by building a machine learning pipeline to predict flight errors in simulator flights. By accurately quantifying errors, the burden of the time-consuming duties of instructors can be reduced, allowing them to focus instead on personalized training for student pilots. Leveraging physiological and aircraft data recorded during simulator flights involving aircraft landings, this research uses functional data analysis, modern feature-selection techniques, and efficient tree-based machine learning methods to predict student performance and highlight key predictive physiological modalities. The proposed pipeline, coined the Multimodal Functional Learning for Error Regression (mFLYER), produces various models that demonstrate notable improvements over prior approaches, closing the gap between successful manual and automated grades. Finally, to further examine the applicability of such models, this approach leverages uncertainty quantification through calibrated models, then ensembles, to identify their ability to confidently differentiate between low- and high-error landings, which many models struggle to accomplish. Finally, additional analysis is performed to determine how future research may improve model performance and confidence.