Simulating video conferences in mixed reality : developing a prototype for research purposes

Many meetings and events take place online via video conferencing platforms. Video conferencing platforms, however, present a major drawback: they condense the event to a single screen, possibly affecting the cognitive ergonomics of the event participants. A possible solution to this problem is mixed reality technology, which provides an entire room, instead of a single screen, as ‘screen space’. To investigate the affordances of video conferences in a mixed reality format, a prototype application for the Hololens2 was developed. Different functionalities were implemented to simulate an academic online conference with separate holograms for the speaker video, the slide show, other conference attendees and the conference program. The position, size and inclination of each hologram could be adjusted to the user’s needs, started and stopped using gesture commands. The app was tested with nine participants. Results from observations and post-task interviews with the participants suggested that the minimal requirements were satisfied, but four issues remained: 1) unreadable slides due to low image resolution, 2) frustration caused by the desire to align the holograms and the imprecision of the gesture commands, 3) need for increased communication and clarification due to the lack of access to the participant’s visual experience, and 4) loss of data due to difficulties in handling Hololens2’s webcam application. The issues were addressed in different ways, including reproducing the slides in better quality, refining the instructions for participants, as well as crafting a paper mock-up to secure the data collection.