A Novel Approach to Material Detection for Blind Amputee Users With Hand Prosthetic

The lack of sensory feedback in upper limb prostheses restrains multiple disability individuals (MDIs) from interacting effectively with their environment. This study proposes a human-computer interaction (HCI)-based framework that uses auditory feedback to overcome visual limitations in recognizing material of objects with a customized hand prosthesis, using a material detection sensor (MDS) to record acoustic and reaction force signals simultaneously. We use clustering techniques to group acoustic data based on reaction force signals clustered (C). The constant amplitude is converted into instantaneous amplitude by calculating each cluster’s mean peak envelope (MPE). We then apply an absolute fast Fourier transform (AFFT) to transfer the signal into the frequency domain. Finally, acoustic features are extracted, specifically, Mel frequency cepstral coefficients (MFCCs), fed into the artificial intelligence (AI) system. The study evaluates ten participants: three upper limb amputees, two blind amputees MDIs, and five nondisabled individuals as the control group. The results demonstrated that the C-MPE–AFFT-MFCC bidirectional long short-term memory (BiLSTM) algorithm outperforms more accuracy than artificial neural network (ANN), linear discrimination analysis (LDA), support vector machine (SVM), and convolutional neural network (CNN) with no significant difference between groups in classification accuracy (CA) (P-value =0.0105). MDIs and the control group classified 30 objects with 92.46% and 97.57% accuracy, respectively, corresponding to a 94.97% reduction in execution time compared with conventional methods. The proposed method could greatly enhance MDIs’ ability to use sophisticated hand prostheses.