Detection of Elderly Falls in Video Streams Using Skeleton Key Points and Transformer Networks

Falls among elderly individuals can lead to significant physical injuries, increased medical costs, and a diminished quality of life. In aging societies, particularly in countries such as Japan, Germany, and Italy, the need for effective fall detection systems is critical. Detecting falls automatically using computer vision techniques presents a notable challenge in the healthcare domain. This study proposes the development of a fall detection system utilizing a Transformer layer with self-attention mechanisms. Our algorithm processes a continuous stream of video frames and classifies whether the accumulated sequence of 44 frames from the video is a fall or not. We employ the MediaPipe Pose framework to extract skeleton key points for person detection. The network is then trained to learn the sequence of movements from these skeleton key points, enabling it to distinguish various types of movements and identify the temporal patterns associated with falls. To further enhance the efficiency and compactness of the model, we incorporate Fnet and MLP-Mixer architectures, which reduce the computational complexity while maintaining high performance. Our proposed model was trained on the NTU RGB+D dataset and demonstrated the capability to differentiate falls from everyday actions with 99.4% accuracy with 1,863 parameters. We also created a baseline by training the state-of-the-art model on the same dataset as ours, achieving an accuracy of 96.3% with 1,714,177 parameters.