Fine-tuning stable diffusion for generating 2D floor plans using prompt templates

Automated generation of 2D floor plans is crucial for architectural design, requiring models to balance precision and adaptability to user-defined specifications. Diffusion models, like Stable Diffusion, excel at generating high-quality images but lack an intrinsic understanding of structured layouts such as floor plans. Conversely, Graph Neural Networks (GNNs) are adept at encoding relational data, representing floor plan objects as nodes and their connections as edges, but they are not generative or capable of processing textual inputs. In this work, we fine-tune Stable Diffusion 1.5 on a custom dataset of floor plans, leveraging structured prompt templates to constrain the model's creativity and guide it toward generating concise, error-tolerant outputs. This research suggests integrating the generative capabilities of diffusion models with the representational strengths of GNNs to overcome inherent challenges in diffusion models, like their inability to explicitly encode spatial relationships. This integration could expand the capabilities of these models, empowering them to comprehend and produce structured layouts more effectively. While computational constraints limited our exploration of this hybrid architecture, our results demonstrate that prompt engineering and dataset preprocessing significantly improve the output quality. This study highlights the potential for generative models in architectural tasks and lays the groundwork for integrating logical reasoning into diffusion-based architectures.