SceneCrafter: Toward Intent-Following Human-AI Collaboration in VR-Based 3D Scene Prototyping

Exploratory 3D scene prototyping is critical in digital media, yet existing tools are limited for early-stage ideation. Although VR enables immersive visualization and embodied spatial interaction, current input techniques fail to capture vague and evolving intent. Recent AI advances enable rapid 3D creation, but existing VR–AI integrations provide week support for coherent creative flow and open-ended exploration. We present SceneCrafter, a VR authoring tool for intent-following human–AI collaboration, where AI tracks users’ evolving intent and adapts responses within a shared immersive environment. It integrates intent cues capturing, dynamic memory updating, intent and spatial reasoning, and in-situ feedback generation to support iterative prototyping. A user study with 16 participants shows SceneCrafter improves usability, lower workload, and human–AI collaboration.