Robotic Cell Micromanipulation Skill Learning via Imitation‐Enhanced Reinforcement Learning

ABSTRACT Humans can learn complex and dexterous manipulation tasks by observing videos, imitating and exploring. Multiple end‐effectors manipulation of free micron‐sized deformable cells is one of the challenging tasks in robotic micromanipulation. We propose an imitation‐enhanced reinforcement learning method inspired by the human learning process that enables robots to learn cell micromanipulation skills from videos. Firstly, for the microscopic robot micromanipulation videos, a multi‐task observation (MTO) network is designed to identify the two end‐effectors and the manipulated objects to obtain the spatiotemporal trajectories. The spatiotemporal constraints of the robot's actions are obtained by the task‐parameterised hidden Markov model (THMM). To simultaneously address the safety and dexterity of robot micromanipulation, an imitation learning optimisation‐based soft actor‐critic (ILOSAC) algorithm is proposed in which the robot can perform skill learning by demonstration and exploration. The proposed method is capable of performing complex cell manipulation tasks in a realistic physical environment. Experiments indicated that compared with current methods and manual remote manipulation, the proposed framework achieved a shorter operation time and less deformation of cells, which is expected to facilitate the development of robot skill learning.