Transformer-based multitask assist control from first-person view image and user’s kinematic information for exoskeleton robots

Exoskeleton robots necessitate the capacity to promptly generate appropriate assistance for the user’s needs across a range of motion scenarios. In this study, we developed a multitask assistance control strategy using a transformer that generates control commands to the exoskeleton robot according to the user’s status and the environment. Our approach captured the user’s joint and trunk motion and a first-person view image as input to the control system. A series of motion tasks were employed to validate the implemented AI with the proposed approach, including walking, squatting, and a step-up movement. Healthy subjects participated and the application of AI with our method to an exoskeleton robot reduced muscle loads. Moreover, the learned assist strategy was found to generalize, reducing muscle activity in another participant. These findings represent a first step toward achieving exoskeleton robot control that assists diverse movements across individuals in various environments using our transformer-based approach.