Visual Generative Artificial Intelligence (GenAI) in Mixed Reality (MR): A Survey of Techniques, Interaction Designs, and Evaluations
GenAI is increasingly reshaping spatial interactions in immersive environments, bridging diverse input and output modalities to create intelligent user experiences. This survey provides a structured examination of visual GenAI techniques adapted and applied to MR contexts, which include Virtual Reality (VR), and Augmented Reality (AR), emphasizing their roles in enabling rich, adaptive, and naturalistic interactions. We screened publications from international venues between 2018 and 2025, followed by an in-depth review of 102 papers. We focus on the applications of visual content creation in 2D&3D and vision-language models (VLMs), highlighting their integration practices within MR. Furthermore, we investigate the co-evolution of generative models and MR interaction design, including immersive visualization, controllable generation, and synthetic data creation. The survey concludes by outlining recent challenges and potential research directions to advance the synergy of visual GenAI and MR techniques.
Authors
- Linjie Qiu (ORCID: https://orcid.org/0009-0008-9204-5690)
- Mingming Fan (ORCID: https://orcid.org/0000-0002-0356-4712)
- Yue Jiang (ORCID: https://orcid.org/0000-0003-0022-6512)
- Duotun Wang (ORCID: https://orcid.org/0009-0005-4393-5230)
- Zeyu Wang (ORCID: https://orcid.org/0000-0001-5374-6330)
- Linlin Cai (ORCID: https://orcid.org/0000-0002-0264-3953)
Institutions
- Hong Kong University of Science and Technology (HK)
- University of Utah (US)
- Machine Science (US)
- University of Hong Kong (HK)
Publication Details
- Journal
- International Journal of Human-Computer Interaction
- Published
- 2026-08-18
- DOI
- https://doi.org/10.1080/10447318.2026.2713822
- Primary Topic
- Multimodal Machine Learning Applications
- Type
- article
- Field-Weighted Citation Impact
- 0.00