Sparse Millimeter-wave Radar Point Clouds based Transformer for Human Activity Recognition

Compared to visual and wearable sensors, activity recognition based on millimeter-wave radar offers a better experience in terms of privacy and comfort. However, the point clouds generated by the millimeter-wave radar are sparse, and existing methods struggle to extract temporal and spatial features from the point clouds. To fully exploit the information in sparse point clouds sequences and accurately capture the spatiotemporal relationships within them, we propose Millimeter-wave radar-based Transformer for human Activity Recognition (MTAR), an innovative approach that achieves accurate human activity recognition by leveraging sparse point clouds as input through Transformers. This method enhances the identification and feature extraction of key channels in point clouds data by introducing PointNet with channel attention mechanisms. Additionally, we employ a dual-branch Transformer architecture, specifically designed to analyze the intrinsic connections within the temporal and spatial dimensions, and further optimize the integration of temporal and spatial features through an adaptive-weighted fusion Transformer, thereby improving the overall performance of the model. Comprehensive evaluations show that with fewer parameters and similar FLOPs, the MTAR outperforms the closest rival by 2.5% on the MM- Fi dataset and by 2.35% on the MMgesture dataset, showcasing its superior generalization performance in experiments with different individuals and environments as well.