A Hybrid Learning Framework for Efficient and Stable Robotic Peg-in-Hole Assembly via PPO-BC

Traditional deep reinforcement learning (DRL) exhibits training inefficiencies in complex robot assembly tasks. Behavior Cloning (BC) can alleviate this issue by providing expert guidance. However, using BC alone often leads to policy instability. This paper proposes a Proximal Policy Optimization with Behavior Cloning (PPO-BC) framework for robotic peg-in-hole assembly. It is a hybrid learning framework that strategically combines BC-guided policy initialization with progressive reinforcement learning. The framework first leverages expert demonstrations to inject prior knowledge through BC pretraining, effectively reducing initial random exploration. Subsequently, to ensure stable convergence and make full use of expert data, a hybrid training strategy is adopted. It progressively blends expert demonstrations with environmental interactions. This helps maintain policy alignment with expert behaviors while enabling adaptive learning. Experimental validation on a UR5 robotic platform demonstrates the superiority of PPO-BC. It achieves 24.8-hour faster convergence, 18 fewer assembly steps, and a 99% success rate.