GRAsPAD: Generalized framework for optimal grasp key-points active detection

In the challenging context of human-robot collaboration, the ability to capture critical human-grasp-related features in a short period is essential for the transfer of skills/experience and the effective imitation learning of human grasping strategies to robotic systems. The paper at hand introduces a novel system for pose optimization of an active (moving) sensor, reducing uncertainty in grasp perception and enabling a more accurate observation of the human grasp pose. The proposed framework, coined as "GRAsPAD" (Generalized fRAmework for optimal graSp key-Points Active Detection), leverages a keypoint detection model to identify grasp-related features in a tomato fruit. The camera pose is iteratively adjusted through the Covariance Matrix Adaptation with Margin - Evolution Strategy (CMAwM-ES) to minimize uncertainty in detected keypoints, enhancing the accuracy of grasp perception and ensuring a clearer, more stable grasp representation. To mitigate sensor noise and variability, a Kalman filter is applied to refine keypoint detections over time. The experimental results demonstrate the significant improvement achieved by the pose optimization of the observer, in terms of grasp perception quality, which is critical for robotic skill learning and effective transfer of human grasping strategies to robotic systems.