High-Accuracy Acquisition of Both Hands’ Postures for Multiple Users Using Ultrafast Mirror-Drive Pan-Tilt Camera

We propose a method for acquiring accurate posture and position of both hands for multiple users even from a distance using a system that consists of an ultrafast mirror-drive pan-tilt camera and a wide-view camera. By performing multithreaded high-speed gaze control for each target, the ultrafast pan-tilt camera can function as multiple virtual telephoto pan-tilt cameras. A wide-view camera is used to detect the positions of the user's hands, and each of the virtual telephoto pan-tilt cameras is utilized to obtain the images around the hands with high spatial resolution. By using these images, it is possible to achieve high-accuracy hands skeleton detection and obtain accurate hand postures. In addition, by synchronizing the whole body posture with the accurate hand postures and by correctly integrating the information obtained between different cameras, it is possible to obtain accurate hand positions. In the experiment, we acquired hand postures from a user standing 5 meters away from the camera system and classified them using an algorithm focused on the extension motion of each finger. As a result, our proposed method achieved an accuracy of 81.0% in classifying hand postures.