Text semantics to controllable design: A residential layout generation method based on stable diffusion model

Controlling flexibility and design effectiveness is challenging in artificial intelligence-based residential layout design. Compared to the previous rule-based or graph-based generation methods, this paper proposes a controllable multimodal approach based on the Stable Diffusion model for generating residential layouts. This method incorporates natural language as design constraints and introduces ControlNet, which enables the generation of controllable layouts through two distinct pathways. A knowledge graph and natural language mapping scheme is proposed to provide an interpretable representation of the design knowledge encapsulated in the knowledge graph. The comprehensibility of natural language and the diversity of input options enable professionals and non-professionals to directly express design requirements, thereby providing a flexible and controllable design method. Finally, comparative and ablation experiments verify the controllability and diversity of the two proposed design paths under multimodal constraints. • Proposed a natural language representation for residential layout design rules. • Introduced a controllable multi-modal generation approach to residential layout design. • Proposed learning design rules by large model fine-tuning techniques. • Experimentally confirmed the controllable efficacy of natural language constraints.