Simulation Environment Generation Using a Persona-aware LLM

Collecting diverse real-world data for AI model training is often costly, time-consuming, and constrained by safety and experimental conditions, making simulation environments an important alternative for data collection. However, existing methods for generating simulation environments often rely on predefined asset libraries and provide limited support for integrating user-specific personas. This paper proposes a persona-aware pipeline for generating 3D desk simulation environments based on a user persona defined by an occupation and Big Five personality traits. Given the input persona, an LLM generates structured scene layout information and an image generation model produces a synthetic desk image, followed by library-free 3D asset reconstruction using SAM 2 for object segmentation and SAM 3D for object-level 3D reconstruction. The assets are then rendered in Isaac Sim using grid mapping and scene graph-based spatial relationships. Experimental results show that persona variables produce interpretable changes in object composition and workspace atmosphere. Qualitative rendering results further show that the proposed pipeline can transfer the layout structure of a target image into a simulation environment, demonstrating its potential for generating diverse and controllable persona-aware simulation environments.