Arm Gesture Recognition with Smartwatches

The use of wearable devices with inertial sensors is on the rise. Research has confirmed that these sensors, through machine learning, can identify actions and recognize users' identities. Recently, a system for recognizing arm gestures has been proposed, based on a custom bracelet equipped with an ESP8266 microcontroller (MCU) and an Inertial Measurement Unit (IMU), connected to a cloud server that processes the inertial data using a Deep Learning model based on recurrent neural networks. The current trend is to move computing towards edge nodes with the introduction of new neural microprocessors thus reducing the need of cloud computation for mobile devices. However, the limited computing power of the MCU used in the state-of-the-art system prevents the autonomous execution of the Deep Learning model, necessitating the support of the cloud server. This study aims to adapt the gesture recognition system proposed in the literature to work autonomously on a consumer wearable device with Wear OS. The device is equipped with a Qualcomm 5100 microprocessor and a Cortex M33 co-processor. The project involved first transferring the algorithm to Wear OS using TensorFlow Lite, followed by the collection of new arm gesture data to adapt the model to the specifications of the wearable device's IMU. The results showed a slight decrease in accuracy compared to the original system, primarily due to the new data set being smaller in size than the state of the art. However, the computational efficiency of the wearable device makes it very promising for augmented reality and remote control applications.