Biopotential-Informed Reward Inference and Constraint Design for Consecutive Upper-Limb Motor Coordination Tasks in Robot Manipulation

In consecutive upper-limb motor coordination tasks, where a new condition is imposed once a subgoal is achieved, kinematic demonstrations reveal what a human does but not how cautiously, so existing imitation learning misses the adaptation to the new condition. To close this gap, we propose designing reward and constraint functions from biopotential-informed characteristics of human surface electromyography (sEMG). The task begins as a Markov decision process (MDP) and turns into a constrained MDP (cMDP) once the first subtask is completed. The sEMG data are decomposed into shared and task-specific muscle synergies; the former infers a quadratic-regulator-formed reward by inverse optimal control, and the latter defines a subtlety coefficient bounding the action magnitude in the constrained phase. A two-holes two-balls patience cube exemplifies the task, and the policy is trained by constrained policy optimization and deployed on a robot manipulator. Whereas unconstrained PPO and SAC agents complete the constrained phase in only 1% of the episodes, the proposed agent completes 100% with the shortest completion time (1.76 s) among four constrained variants and most closely reproduces the human’s dwell-time distribution and zone transitions near the hole, yielding a feasible and human-like policy.