Multi-Head Spatio-Temporal Transformer with Context-Aware Fusion for Human Activity Detection Using IoT Sensors

Currently, Human Activity Detection (HAD) through the Internet of Things (IoT) has been developed as a crucial component in several applications, such as healthcare monitoring and behavior analysis. The existing Convolutional Neural Network and Long Short-Term Memory (CNN-LSTM) utilize IoT sensors, although they struggle to accurately identify dynamic activities such as walking owing to overlapping signal patterns and lack of contextual awareness. Hence, this study proposes a Multi-Head Spatio-Temporal Transformer with Context-Aware Fusion (MST-CAF) to effectively detect dynamic activities in complex environments. Initially, the input data were collected from opportunity and extrasensory datasets by capturing a variety of human activities in real-world scenarios. Then, Butterworth and median filters, as well as Blackman windowing, are employed to remove noise from raw signals and preserve essential signal characteristics and segment signals, respectively. Subsequently, feature extraction was performed to extract handcrafted statistical, signal-based, and frequency-domain features. Further, the sensor and external context features are fused using Multi-Head Attention (MHA) to generate enriched embeddings. Finally, a spatiotemporal transformer is employed to capture the long-term dependencies across activity sequences and accurately recognize human activity classes. The proposed MST-CAF attained better results in terms of accuracy (0.9718) than the existing Time-Convolution Network with the Attention Mechanism (TCNattention) model.