Utilizing Generative AI for Patient Behavioral Assessment

We propose a vision-language framework designed for clinical environments to monitor patient behavior and generate clinically informative reports using advanced computer vision and large language models (LLMs). Our framework analyzes camera images, identifies patient's postures, activities, and interactions with objects, and provides insights into a pretrained summarization LLM to produce context-aware reports. This is done by leveraging a custom-trained Convolutional Neural Network (CNN). This vision-language approach enables real-time or offline monitoring and report generation without the need for wearable devices, offering a non-intrusive, automatic, and scalable monitoring for healthcare settings.