A Two-Stream Hybrid GCN-Transformer Fusion Network for Elderly Activity Recognition

Elderly Activity Recognition has become increasingly critical due to the growing number of elderly individuals living alone, making them vulnerable to various risks. This study focuses on skeleton-based activity recognition tasks for elderly, which are gaining popularity due to the robustness of skeletal data as well as less computational effort. However, elderly activity recognition is often challenged by human-object interaction actions and similar actions. For this reason, we approach the problem from a network perspective and a data perspective, respectively. From a network perspective, we consider combining the GCN network and the Transformer network to improve the accuracy and robustness of the recognition through the complementarity of the two at the network level. From the data perspective, we design a fusion module to further integrate the joints data and bones data to explore the more detailed information contained in the data, so as to better meet the challenges of the elderly activity recognition task. In this work, we propose a Two-stream Hybrid GCN-Transformer Fusion Network (THGTF-Net), which combines the GCN and Transformer networks while deepening the fusion of joints data and bones data to better handle the elderly activity recognition task at the network level as well as the data level. Finally, experiments on the largest-scale elderly activity dataset, ETRI-Activity3D, demonstrate the effectiveness of our proposed method. Additionally, we validate our method on the standard activity recognition dataset NTU RGB+D, achieving excellent results that demonstrate the universality of our approach.