Privacy-aware fall detection and alert management in smart environments using multimodal devices

Falls are a leading cause of injury and mortality, especially among the elderly. While camera-based fall detection systems have shown success, they raise significant privacy concerns. Alternatives using wearable sensors or thermal cameras offer comparable accuracy but have yet to be combined for accurate fall detection. Additionally, most research focuses on fall detection without addressing post-fall user’s condition or personalized alerts. This study aims to develop a privacy-aware fall detection system leveraging wearable sensors and thermal cameras. In addition, an alert system integrates devices such as voice assistants and speakers to assess the user’s status after the fall and notify the event. The system improves detection accuracy, addresses privacy concerns, and enhances alert management through personalized responses. We propose an Internet of Things (IoT)-based system integrating all sensors and devices previously mentioned. Edge-based computation enables real-time detection, with Internet connectivity used only for sending alerts in case of a fall. Various machine learning algorithms and sensor sources are evaluated to determine their impact on detection accuracy. Experimental results show that fall detection using a convolutional neural network with thermal images from three viewpoints achieves an F1-score above 0.98. Similarly, traditional machine learning algorithms applied to wearable sensor data showed high performance (0.93 F1-score). Post-processing techniques effectively remove false positives, improving reliability and adoption in real environments. The proposed system ensures high accuracy while addressing privacy concerns. By integrating multimodal devices and edge-based computing, it offers a scalable, real-time solution for smart environments, ensuring timely responses through personalized alerts after falls. • Thermal imaging can be used to detect falls while preserving privacy. • Multimodal detection is crucial for selecting the appropriate devices. • Convolutional and recurrent neural networks are appropriate for detecting falls. • A service with a voice assistant and a large language model reduce false positives.