Video-Based Fall Risk Assessment Using Multimodal Large Language Models in Home Health Care

Falls cause millions of injuries and deaths annually, making prevention a key priority in home health care (HHC). Traditional fall risk assessments often overlook the complex interaction of personal, environmental, and behavioral factors. This study addresses these limitations by introducing a novel approach that leverages multimodal data, specifically visual frames and structured prompts, to assess fall risk in in-home patients. Using the multimodal large language model (MLLM), LLaVA-NeXTVideo-7B-hf, we analyze simulated in-home patients' video data to enable a more comprehensive and dynamic evaluation of fall risk, paving the way for intelligent, video-based fall prevention in home health care. Preliminary validation using simulated video data demonstrates the feasibility of using MLLMs for such tasks. Simulated in-home patient video data were processed into 24 equally spaced frames. Twelve visually observable fall risk factors extracted from the literature search, categorized as intrinsic, extrinsic, or behavioral, guided the creation of prompts for the MLLM. Standardized prompts were developed by testing the model with concise prompts for simple inferences and elaborated prompts for complex ones. Each prompt was run 3 times, and consensus results were compared with expert evaluations. The model achieved 85.71% accuracy with concise prompts on 7 simple risk factors and 100% accuracy with elaborated prompts on two complex ones. However, the model consistently failed for 2 risk factors that required clinical judgment or had limited visual data. MLLMs like LLaVA-NeXTVideo 7B-hf show strong potential for augmenting fall risk assessment in HHC when guided by well-structured prompts. The approach focuses on visually inferable factors and is intended to complement, rather than replace, clinical evaluation. This proof-of-concept feasibility study shows that MLLMs can support preliminary fall risk analysis using simulated home health care video data and lays the groundwork for future video-based research in this setting, where existing work remains limited. To our knowledge, this is the first study to evaluate the feasibility of MLLM-based video analysis for fall risk assessment in home health care.