Human Activity Recognition by Fusing Ultrawideband Radar and Low-Resolution Infrared Information

The recognition of continuous daily human action is of great significance in smart homes, health management, and other areas. It is the key to achieving “Big Health” model. Both thermopile infrared array sensor (TPAS) and impulse radio ultra-wide band (IR-UWB) radar can detect human target and provide relative rich information without privacy invasion. This paper builds a hardware platform based on these two types of sensors, collects data from actual indoor scenes, and proposes a human action recognition algorithm for achieving human action segmentation, human target localization and recognition of multi-level activities. A human action segmentation algorithm is proposed based on finite state machine (FSM) and R-square value. Three data fusion schemes are designed based on CNN network: data level fusion, feature level fusion, and decision level fusion to recognize human actions. A positioning algorithm is proposed fusing TPAS and UWB information to determine orientation and distance. Finally, a scene model is established containing location, objects, spatial and temporal information, and Hidden Markov Model (HMM) is trained to encode the information and predict high semantic human activities. The experimental results show that the accuracy for human dynamic recognition reaches 97.22%, the success rate of human positioning achieves 92.03% and the system can accurately recognize continuous human actions and high semantic activities in actual scenes.