A Deep-Learning-Driven Aerial Dialing PIN Code Input Authentication System via Personal Hand Features

The dialing-type authentication as a common PIN code input system has gained popularity due to the simple and intuitive design. However, this type of system has the security risk of “shoulder surfing attack”, so that attackers can physically view the device screen and keypad to obtain personal information. Therefore, based on the use of “Leap Motion” device and “Media Pipe” solutions, in this paper, we try to propose a new two-factor dialing-type input authentication system powered by aerial hand motions and features without contact. To be specific, based on the design of the aerial dialing system part, as the first authentication part, we constructed a total of two types of hand motion input subsystems using Leap Motion and Media Pipe, separately. The results of FRR (False Rejection Rate) and FAR (False Acceptance Rate) experiments of the two subsystems show that Media Pipe is more comprehensive and superior in terms of applicability, accuracy, and speed. Moreover, as the second authentication part, the user’s hand features (e.g., proportional characteristics associated with fingers and palm) were used for specialized CNN-LSTM model training to ultimately obtain a satisfactory accuracy.