Identification of User's Interests by Deep-Learning Based Pose Estimation with Angle Relationship from Keypoints for Smart Advertising Displays

In this paper, a deep-learning based pose estimation technology for recognizing user's interests on interactive advertising displays is developed by leveraging the YOLO-Pose architecture. The primary goal is to predict user's attention areas by analyzing angular relationships with human body poses. The methodology uses the YOLO-Pose model to identify 17 keypoints on the human body and the angles among the selected body keypoints are calculated. The proposed design incorporates the left and right hand identification and the height determination to address accuracy issues from variations in user's height or body size. By integrating the estimation data for angles, user's height, and handedness, the horizontal and vertical SVM classifiers will be pre-trained. After the SVM classifiers are well trained, the combination results from both SVM classifiers enables the prediction of the user's area of interests. Experimental results with four, six, and eight grid divisions on the display reveals that the average identifying accuracies for horizontal and vertical grid classification can be up to 94% and 83%, respectively. By height adjusted information, the horizontal and vertical classification accuracies of the SVM classifiers can improve by 11% and 20% respectively within corresponding height ranges. For the embedded device implementation, the YOLO model and SVM classifier are deployed on the Jetson AGX Orin platform to achieve an average processing speed of 15 frames per second.