Neural networks empowered: a machine learning-enabled, Gyro mmID for enhanced virtual reality and motion tracking applications

Abstract With the emerging developments in millimeter-wave/5G technologies, the potential for wireless Internet of things devices to achieve widespread sensing, precise localization, and high data-rate communication systems becomes increasingly viable. The surge in interest surrounding virtual reality (VR) and augmented reality (AR) technologies is attributed to the vast array of applications they enable, ranging from surgical training to motion capture and daily interactions in VR spaces. To further elevate the user experience, and real-time and accurate orientation detection of the user, the authors proposes the utilization of a frequency-modulated continuous-wave (FMCW) radar system coupled with an ultra-low-power, sticker-like millimeter-wave identification (mmID). The mmID features four backscattering elements, multiplexed in amplitude, frequency, and spatial domains. This design utilizes the training of a supervised learning classification convolutional neural network, enabling accurate real-time three-axis orientation detection of the user. The proposed orientation detection system exhibits exceptional performance, achieving a noteworthy accuracy of 90.58% over three axes at a distance of 8 m. This high accuracy underscores the precision of the orientation detection system, particularly tailored for medium-range VR/AR applications. The integration of the FMCW-based mmID system with machine learning proves to be a promising advancement, contributing to the seamless and immersive interaction within virtual and augmented environments.