Unconstrained Home Appliance Operation by Detecting Pointing Gestures from Multiple Camera Views

In this paper, we propose a method for manipulating home appliances using arm-pointing gestures. Conventional gesture-based methods are limited to home appliances with known locations or are device specific. In the proposed method, the locations of home appliances and users can change freely. Our method uses object- and keypoint-detection algorithms to obtain the positions of the appliance and operator in real time. Pointing gestures are used to operate the device. In addition, we propose a start gesture algorithm to make the system robust against accidental gestures. We experimentally demonstrated that using the proposed method, home appliances can be operated with high accuracy and robustness, regardless of their location or the user’s location in real environments.