SSL-HWE: Semi-Supervised Learning-Based Hierarchical Waypoint Extraction for Imitation Learning in Robotic Manipulation

Imitation learning has proven effective for performing manipulation tasks when trained on high-quality human demonstrations. However, human demonstrations always contain imperfections that complicate the action data distribution, leading to compounding errors during policy deployment. In this paper, we propose a novel data processing approach to simplify action data for human demonstrations with a small set of labeled key intervals that capture critical actions. The approach integrates two components: semi-supervised key interval segmentation and hierarchical optimization for waypoint extraction. The semi-supervised learning model identifies key intervals in the unlabeled dataset, enabling the segmentation of each trajectory into key and non-key intervals. These interval labels are then used to solve optimization problems with interval-specific constraints to derive waypoints, which are subsequently applied to adjust the actions in the demonstration dataset. The proposed data processing approach was validated through a series of real-world experiments in conjunction with a state-of-the-art imitation learning method. The results demonstrate that our approach significantly enhances downstream imitation learning, and outperforms the compared data processing method by 30% to 50% in success rate. Videos are available at https://liuxinzi.github.io/SSL-HWE.