Liveness Detection Based on Eye Blinks and Pose Estimation Utilizing MediaPipe Deep Learning Model

Along with the development of artificial intelligence and computer vision, there is a need for a module that is capable of processing tasks related to biometrics. In certain cases, such as verification or authentication involving the facial recognition process, a module is needed that is specifically capable of taking a selfie before the verification or authentication process can be carried out. To prevent the selfie module from capturing static input such as photos or dynamic input like videos that have been previously saved, the module requires additional processing, namely liveness detection. In this study, a combination of the use of eye blinking and pose estimation for liveness detection in the selfie module is proposed. The eye blinking detection and pose estimation utilize MediaPipe deep learning model. In the developed module, to take a selfie, the user must first demonstrate a certain pose. When the pose demonstrated by the user is correct, the next selfie process can be carried out by the user by blinking several times. The test results show that the developed module can validate the pose demonstrated by the user, can take a selfie with eye blinks, and can store the entire frame or bounding box of the face and bounding box of the hand that have been successfully captured by the camera. The developed liveness detection module can tackle the spoofing attack with accuracy of 96.67%.