Development of Dementia Care Training System Using AR and Large Language Model

We have developed HEARTS, a dementia care training system using augmented reality based on Humanitude. Humanitude is a multimodal comprehensive care technique for dementia, and has attracted attention as a method to reduce the burden on both caregivers and patients. However, the HEARTS developed so far could not evaluate “speaking” skills based on the content of conversations among “seeing,” “touching,” and “speaking,” all of which are fundamental skills in Humanitude. Therefore, we attempted a new quantitative evaluation of trainees' “speaking” skills by estimating the emotional value of conversational content using GPT-4 named HEARTS 5. A survey of caregivers was conducted using the developed system and was well received by the participants. We also developed a HEARTS 5 conversational version based on HEARTS 5, with the addition of GPT-4 conversation generation and Azure Text-to-Speech.