mint: Integrating scientific visualizations into virtual reality

Abstract We present an image-based approach to integrate state-of-the-art scientific visualization into virtual reality (VR) environments: the mint visualization/VR inter-operation system. We enable the integration of visualization algorithms from within their software frameworks directly into VR without the need to explicitly port visualization implementations to the underlying VR framework—thus retaining their capabilities, specializations, and optimizations. Consequently, our approach also facilitates enriching VR-based scientific data exploration with established or novel VR immersion and interaction techniques available in VR authoring tools. The separation of concerns enables researchers and users in different domains, like virtual immersive environments, immersive analytics, and scientific visualization, to independently work with existing software suitable for their domain while being able to interface with one another easily. We present our system architecture and inter-operation protocol (mint), an example of a collaborative VR environment implemented in the Unity engine (VRAUKE), as well as the integration of the protocol for the visualization frameworks Inviwo, MegaMol, and ParaView. Our implementation is publicly available as open-source software. Graphical abstract