Prediction of Remote Photoplethysmography Using Quaternion-based Convolutional Neural Networks in Metaverse

In Metaverse, where digital and physical realities converge, cameras serve as highly accessible sensors, crucial for non-invasive health monitoring. Remote photoplethysmography (rPPG) utilizes these devices to measure vital signs, a key component for enhancing user health and safety in virtual environments. However, the reliability of rPPG is compromised by user movemeats and variable lighting conditions. To address these challenges, this paper introduces a pioneering quaternion-based convolutional neural networks (Q-CNNs) approach, optimized for the unique data properties of video streams in virtual worlds. By employing quaternion algebra, our model processes the multidimensional data from a camera more effectively, mitigating motion and lighting artifacts and improving the extraction of physiological signals. We detail a methodology encompassing simulation of virtual environments, an optimal neural network architecture, and comparative analysis against conventional CNN frameworks. Our findings demonstrate that Q-CNNs significantly enhance the accuracy and robustness of rPPG signals in a range of Metaverse settings. This advancement not only underscores the potential of cameras as effective health monitoring tools in immersive platforms, but also paves the way for their broader application in real-time medical diagnostics.