Enhancing and Optimizing User-Machine Closed-Loop Co-Adaptation in Dynamic Myoelectric Interface

Co-adaptation interfaces, developed through user-machine collaboration, have the capacity to transform surface electromyography (sEMG) into control signals, thereby enabling external devices to facilitate or augment the sensory-motor capabilities of individuals with physical disabilities. However, the efficacy and reliability of myoelectric interfaces in untrained environments over extensive spatial range have not been thoroughly explored. We propose a user-machine closed-loop co-adaptation strategy, which consists of a multimodal progressive domain adversarial neural network (MPDANN), an augmented reality (AR) system and a scenario-based dynamic asymmetric training scheme. MPDANN employs both sEMG and Inertial Measurement Unit (IMU) data using dual-domain adversarial training, with the aim of facilitating knowledge transfer and enabling multi-source domain adaptation. The AR system allows users to perform 10 holographic object repositioning tasks in a stereoscopic mixed reality environment using a virtual prosthesis represented as an extension of the residual limb. The scenario-based dynamic asymmetric training scheme, which employs incremental learning in MPDANN and incremental training in the AR system, enables the continuous updating and optimization of the system parameters. A group of non-disable participants and two amputees performed a five-day offline data collection in multiple limb position conditions and a five-day real-time holographic object manipulation task. The average completion rate for subjects utilizing MPDANN reached ${83}.{37}\% \pm {2}.{50}\%$ on the final day, marking a significant improvement compared to the other groups. These findings provide a novel approach to designing myoelectric interfaces with cross-scene recognition through user-machine closed-loop co-adaptation.