AIMERS: an AI-based MR scene design system with human-centric perception optimization

Visual realism is fundamental to convincing Mixed Reality (MR) experiences. However, current design workflows implicitly assume that physically-based rendering parameters naturally lead to perceptually realistic results. We begin from the opposite hypothesis: physically accurate parameters and users’ perceived realism are often misaligned, leading to inconsistent visual fusion and significant design overhead. To address this problem, we present AIMERS, an AI- and perception-guided MR scene design framework. First, AI-based neural inverse rendering is used to automatically estimate lighting-independent material, geometry, and illumination properties from multi-view RGB inputs, removing the need for manual material calibration. We then introduce an interactive MR perceptual interface that allows users to adjust key realism parameters during immersive viewing, enabling us to capture perception-aligned preferences across scenes. By jointly analyzing physically derived parameters and perceptual data, we derive optimal parameter intervals that best match perceived realism across different scene categories. Controlled user studies reveal a consistent mismatch between physical correctness and human perception, and demonstrate that combining AI estimation with perception-guided adjustment leads to more coherent and convincing MR visual fusion. Overall, this work establishes a perception-aligned paradigm for MR scene design, bridging the gap between physical accuracy and human perception and providing practical guidance for building realistic MR applications.