Investigating sensors and methods in grasp state classification in agricultural manipulation

Accurate grasp state classification is essential for effective and efficient agricultural manipulation and harvesting. The agricultural environment presents unique challenges, including clutter, occlusion, and the physical attachment of fruit to plants, which necessitate precise separation during harvesting. This study investigates the integration of multiple sensor modalities, including inertial measurement units (IMUs), infrared (IR) reflectance, tension sensors, tactile sensors, and RGB cameras, into a compliant gripper to classify grasp states. We evaluate the individual contributions of each sensor and compare the performance of two widely used classification models: Random Forest and Long Short-Term Memory (LSTM) networks. While most sensors offer only marginal improvements in detecting grasp anomalies such as slip and failure, the inclusion of a tension sensor significantly enhances the detection of sustained successful picks, improving classification accuracy from 10 percent to 80 percent. Furthermore, we identify a minimal viable sensor combination consisting of IMU and tension sensors that achieves robust grasp state classification. This enables real-time feedback for planning corrective actions, thereby improving the reliability and efficiency of autonomous fruit harvesting operations.