A Conceptual Framework for Immersive Behavioral Data Exploration Assisted by a Deep Learned Representation of Data

By allowing to conduct experiments involving eco-logically valid tasks within controlled environments, Virtual Real-ity (VR) offers novel opportunities for studying human behavior. Several modalities can be leveraged, including event logs, motion trajectories, eye-tracking data, or physiological signals. However, analyzing such multimodal data presents considerable challenges due to their inherent complexity, the varied structures they imply, as well as the necessity to use exploratory approaches. There is therefore a need to design visual analytics tools for the exploration of immersive behavioral data without prior knowledge. Our idea is to integrate deep learned computational models -which leverage advanced techniques to extract valuable high-level features from unlabeled data- into exploratory Visual Analytics tools. We introduce a conceptual framework for integrating deep learning models into the Visual Analytics process of immersive behavioral data analysis, also focusing on the services that such systems should provide to the analysts.