Real-Time Hand Gesture Recognition System with User Identification for Industrial Applications

Human-Machine interaction plays a pivotal role in contemporary industrial and civil environments. Among other sensing technologies, Leap Motion Controller is a non-contact and reliable one to be used for gesture recognition. In this paper, a dataset composed of more than two thousand samples is presented. The data have been collected by 17 subjects performing multiple repetitions of 26 gestures based on the Italian sign language alphabet. The “LEAP-SDGest: Leap Motion static & dynamic gestures” dataset can be adopted for the development of algorithms for both gesture recognition and user identification, for example, in industrial control applications. An architecture leveraging the MQTT protocol is proposed for this purpose, embedding gesture recognition and access control by exploiting classifier models and MQTT topics structure. In order to validate the dataset and proposed methodology, Machine Learning and Deep Learning models for both control tasks were developed. After a preliminary analysis, the Multi-Layer Perceptron proved to be the best solution and was optimized through grid-search. The resulting models reached 91.6% accuracy for user recognition and 97% accuracy on gesture recognition through very lightweight models and reduced pre-processing. Under strong authentication conditions, an end-to-end success rate of 91% was achieved for the entire system that includes access control and user recognition.