Pilot gaze intent classification using BiLSTM and gated transformer model

The pilot's gaze intention refers to the purpose and intention reflected by the target or area where the pilot's eyes are focused when performing a flight mission. Its accurate classification not only improves mission execution efficiency and flight safety, but also provides key support for optimizing the flight experience. For pilots Gaze intention data are usually characterized by multi-feature long time series, and due to the shortcomings of traditional methods, a pilot gaze intent classification method that combines BiLSTM and gated Transformer models is proposed. This method inputs the pilot's time-series-based gaze information on the cabin area under different flight tasks, including gaze coordinates, gaze area, gaze duration, etc., into two improved Transformers, one of which is channel-based and the other It is based on the step size design and is integrated into BiLSTM before and after multi-head attention. Finally, by merging the gating of the two Transformer towers, using linear layers and softmax to output the accuracy of each output intention category, the classification of gaze intentions is achieved. By applying different models to the pilot gaze intention data set for comparative experiments, the results show that the F1 score of the method proposed in this article reaches 94.84%, which has better classification results than other models.