Normalizing Flow-Based Fine-Grained Modeling for Unknown Gesture Rejection in Myoelectric Pattern Recognition

Gesture recognition systems based on surface electromyography (sEMG) exhibit high accuracy in laboratory settings. However, they often underperform in real-world applications due to the occurrence of unknown gestures not encountered during training. Prototype learning methods, which learn gesture prototypes and classify unknown gestures based on distances to these prototypes, effectively reject unknown gestures. However, relying solely on global feature distances may overlook subtle variations, weakening discrimination between similar features and reducing the model’s ability to identify unknown gestures resembling known ones. To address these limitations, we propose a fine-grained method that models the probability distribution of each feature point, enabling the detection of subtle differences in partial features. Specifically, we employ normalizing flows to capture detailed information at the feature-point level. This approach enhances the model’s capacity to recognize challenging unknown gestures that partially differ from known gesture patterns. In addition, we introduce synthetic unknown gestures generated by applying slight perturbations to known samples, simulating challenging unknown scenarios. We then design a novel loss function that pulls known gestures closer together while pushing synthetic unknown gestures further apart, creating a more robust rejection model. Extensive experiments on both custom and public datasets demonstrate that our method achieves an area under the curve (AUC) of 0.988 on the custom dataset and an average AUC of 0.984 and 0.782 on the two public datasets, Capgmyo-dbc and NinaproDB5, respectively. These results indicate that the proposed method provides a robust and practical solution for reliable myoelectric control in real-world applications.