Early Grasp Prediction With Incomplete Data via Spatial Gating and Temporal Weighting for Teleoperation

Accurate and prompt speed grasp intention recognition is crucial in online human-robot interaction (HRI). However, dynamic grasping relying on complete motion for high recognition accuracy will lead to an unavoidable delay in real-time prediction. To address this issue, we propose a Spatial Gating and Temporal Weighting Early Grasp Prediction (STEGP) method that utilizes incomplete dynamic grasping data from sliding windows to reduce the time delay for reliable robot teleoperation. The proposed method comprises a synergy-based feature extraction module, a spatial gating classification module, and a time-decay weighting fusion prediction module. The spatial-temporal mechanism with gating units effectively classifies sequential movements, achieving performance comparable to that of Transformers but being much easier to implement. Integrating a time-decay weighting frame enables reliable early prediction even with incomplete data. gMLP is chosen for the classification of hand dynamic grasping because of its high accuracy, realizing 93.83% accuracy for 33 grasping categories. The prediction tests demonstrated 85.4% accuracy, with the first 25% grasp completion across 28 subjects. Online robotic teleoperation grasp experiments achieved a 57.4% reduction in time delay and a 93.3% success rate.