From 2D to XR: enhancing object placement in design simulations through spatial relations

Object placement is an essential task in extended reality (XR) applications, particularly within design and educational simulations, where realistic positioning enhances immersion and usability. Despite significant advancements in object placement methods across various domains, the literature reveals a gap in the thorough exploration of spatial relations, particularly in image composition and XR. To address this, we enrich the object placement assessment (OPA) dataset by focusing on spatial relations between foreground and background objects. Subsequently, we train a model to predict the position of a foreground object based on its relation to background objects. The model's performance in image composition tasks shows reasonable overlap but struggles with complex cases, indicating the need for improved contextual understanding. In the XR environment, we incorporate scene understanding and depth estimation to refine predictions and place the foreground object appropriately. A human evaluation study further elucidates the performance of the proposed solution across image composition, indoor and outdoor virtual reality (VR) environments, and mixed reality (MR) settings, revealing variability in satisfaction ratings. Additionally, we conducted a downstream task in the field of interior design simulations to further study the implications of object placement based on spatial relations. The results revealed that while the proposed solution was efficient and promising, it also presented initial challenges and required manual interaction for final placement. These findings provide a foundation for enhancing object placement tasks, with implications for VR, MR, and other fields requiring precise spatial understanding.