Brain-Focused Multimodal Approach for Studying Conversational Engagement in HRI

My research adopts an interdisciplinary approach to study conversational engagement in human-robot interaction, by integrating cognitive neuroscience with multimodal behavioral measures and self-assessment, to provide a more comprehensive and objective evaluation of user experience. By utilizing brain imaging to analyze conversations, I aim to investigate the differences between interactions with humans and robots, as well as enhance our understanding of the cognitive mechanisms underlying communication. In addition to exploring variations in neural patterns for different agents, my work leverages multimodal machine learning to assess how brain imaging data, combined with other modalities such as eye tracking, audio, and video, can improve engagement detection, to ultimately design robots that can effectively detect, evaluate, and respond to user engagement, thereby facilitating more effective communication.