[DC] Supporting Asynchronous Collaboration in Mixed Reality with Large Language Models (LLMs)

While prior work has demonstrated the benefits of asynchronous collaboration in Mixed Reality (MR) for tasks like education and industrial training, it is still under-investigated compared to synchronous collaboration. Previous asynchronous MR systems rely mainly on recording and replaying of the collaborators’ actions, which raises many challenges in terms of interactivity, understanding and processing speed of the recorded data. My PhD project investigates how Large Language Models (LLMs) can be used to improve asynchronous collaboration by creating a virtual surrogate which can interact on behalf of a person who is not available at a particular time. This paper presents the research approach, the application context and the first prototype developed as part of this project, as well as the future work planned for the upcoming years.