Utilization of Force Sensor and Deep Q-Learning Algorithm in Object Grasping Simulation Identified by Mask R-CNN

This research focuses on the development and evaluation of a robotic grasping system that combines a force sensor and a Deep Q-Learning algorithm, with objects identified using Mask R-CNN. The system’s performance was tested using 10 randomly selected objects with varying shapes, sizes, and hardness to assess its adaptability to different scenarios. During training, the Deep Q-Learning network was trained over 1000 episodes, showing stable convergence with increasing average reward values per episode. The Random method demonstrated a grasp success rate of 62% and action efficiency of 60%, while the Pre-Train VPG method showed improvement with a grasp success rate of 67% and action efficiency of 65%. The VPG with Mask R-CNN further improved results, achieving a 77% grasp success rate and 76% action efficiency. Our method with binary reward achieved an 81% grasp success rate and 80% action efficiency, while our fully developed method recorded the best results with an 86% grasp success rate and 84% action efficiency. Mask R-CNN maintained an object detection accuracy of 98%, ensuring precise identification for grasping by the UR5 robotic arm. The processing time from object detection to grasp execution was 0.5 seconds per step, enabling real-time operation. The Dynpick force sensor provided accurate feedback on force and torque, allowing for adjustments with a variation of ±5% from the optimal value, crucial for maintaining the integrity of the objects tested. The findings demonstrate that the developed system is effective and efficient in handling various objects, indicating its potential for practical robotic applications. This research highlights the significant contributions of machine learning algorithms and sensors in enhancing robotic performance, paving the way for future advancements in the field.