FatigueSense: Multi-Device and Multimodal Wearable Sensing for Detecting Mental Fatigue

Mental fatigue is a crucial aspect that has gained attention across various disciplines due to its impact on overall well-being. While previous research has explored the use of wearable devices for detecting mental fatigue, limited investigation has been conducted into the effectiveness of these devices in different body positions or in multi-device setups. To address this, our study utilizes a unique public dataset containing over 13 hours of sensor data collected across 36 sessions, with four wearable devices (Earable, Chestband, Wristband, and Headband). We propose several machine learning-based approaches to assess both psychological and physiological mental fatigue levels in a multimodal and multi-device environment. Specifically, we introduce device type-specific approaches (trained and tested on a single device) and multi-device approaches (trained and tested on multiple devices) for mental fatigue inference tasks. Our findings show that device type-specific models perform well, with AUC scores ranging from 0.63 to 0.69 for psychological and from 0.74 to 0.80 for physiological mental fatigue. The multi-device approach shows improved performance for psychological mental fatigue (AUC of 0.69 to 0.74) and physiological mental fatigue (AUC of 0.81 to 0.88). Hence, this study presents a unique and in-depth analysis of detecting mental fatigue with wearables, demonstrating the potential of machine learning-based approaches in multi-device and multimodal setups that are prevalent in today’s emerging lifestyles.