A wearable thumb device for fruit firmness estimation with vision-based tactile sensing

Recent advancements in non-destructive technologies have enabled precise firmness measurement for various fruits, including kiwifruit. However, existing methods remain limited by high costs, environmental sensitivity, and field application impracticality. This work introduces a novel wearable device for estimating non-destructive fruit firmness, combining human tactile interaction with vision-based tactile sensing and edge computing. Worn on the thumb, the device leverages embodied intelligence, merging intuitive human touch with the precision of a vision-based tactile sensor. A single-board computer processes tactile images locally, enabling reliable operation even in remote environments. The device employs our proposed deep learning model for real-time firmness predictions from a single palpation, minimizing repetitive handling and reducing fruit bruising. Its ergonomic, symmetrical design supports comfortable use on either hand, enhancing usability. Compact and portable, the device integrates essential components within a housing measuring 40 mm × 25 mm × 72 mm and weighing only 135 g. Validated through non-destructive ripeness assessments on ’Hayward’ Kiwifruit, the device demonstrated a strong correlation between tactile images and firmness values when paired with our proposed model, achieving a coefficient of determination ( R 2 ) of 0.89. This study created a dedicated dataset on Kiwi firmness to support model development and validation. Moreover, this work’s proposed dataset and source code is available at https://mashood3624.github.io/WearableDevice/ . • A novel wearable device for non-destructive, and real-time fruit firmness estimation is proposed. • A deep learning model is proposed and deployed on the device with a R 2 of 0.89%. • A “Hayward” Kiwi dataset with 530 pairs of tactile palpation and penetrometer firmness readings was collected to validate the device. • The device was validated for real-time firmness, demonstrating its practicality in agriculture.