A method of detection and recognition for static gesture based on Faster-RCNN and dual-streams CNN

For improving the situation of incomplete utilization of gesture features in Static gesture recognition, this paper proposes a dual-streams convolutional neural network model using the RGB/grayscale and black-and-white images to detect and recognize static gestures for adequate learning of the images’ features and improvement of recognition accuracy rate. In this model, the K-means Clustering Method is used to remove the background, the Faster-RCNN model is used to process the image to get the gesture position, and the number of training samples is expanded to enhance the generalization of the training. Finally, two kinds of images are inputted into the network training and gotten integrated before the classification, which make full use of the color texture features and the edge shape features to improve the accuracy of recognition. Verified by Thomas Moselund and Jochen Triesch’s gesture database, the recognition accuracy reached 97.91% and 97.20%.