Optimization of High-Speed Train Driver Gesture Recognition Using Self-Supervised Learning and Stream State-Tying

During train operations, drivers need to maintain a fixed driving posture for extended periods, and due to the monotonous train operating environment, it can lead to a decline in the quality of the driver's work, thereby posing potential safety hazards. According to the high-speed train driver standards, executing prescribed gestures is a crucial aspect of their driving operations. Detecting driver gestures allows for the effective assessment of the driver's operational status and quality, thereby ensuring train operation safety. Traditional manual inspection methods are inefficient and inadequate for meeting actual needs. Existing gesture recognition algorithms suffer from low detection accuracy and require extensive manual annotation of training data. With the development of intelligent railways, building a high-precision, labor-reducing train driver gesture recognition model has become an industry necessity. This paper proposes a three-stage train driver gesture recognition algorithm based on the aforementioned situation. In the first stage, the self-supervised learning (SSL) method MoCo contrastive learning is used to extract vector features from known gesture categories. In the second stage, flow state binding is utilized to jointly model multiple synchronized data streams of gesture direction, position, and angle for gesture category recognition. In the third stage, a Generative Adversarial Network (GAN) is used to generate additional gesture samples for training, addressing the complexity and diversity of recognition environments. The proposed method aims to improve gesture recognition accuracy, reduce manual annotation workload, and enhance gesture recognition accuracy in different environments.