ControlBldg: A variable-controlled generative framework for conditioned modeling of vast 3D urban buildings

Development of urban digital twins critically focuses on modeling three-dimensional (3D) buildings. Although numerous approaches have been proposed for 3D building reconstruction in urban environments, most cannot handle data deficiencies in specific areas, which prevents further improvements into more efficient approaches. While emerging methodologies using artificial intelligence-generated content provide alternative 3D digital cousin models without strict data source requirements, this study derived building digital cousins from it and proposed a generative framework incorporating multiple controlling factors for creating simulated building digital cousin representations as simulated approximations for efficient real-world 3D urban modeling. Our framework uses building footprints as a graphical control and parameter series as an appearance control to approximate building geometries by generating a pixel-wise building height map and then reconstructing the 3D architecture of the second level of details (LoD). This approach fully utilizes abundant pre-trained resources from existing large visual models and yields satisfactory results. In quantitative and qualitative evaluations, our proposed framework achieves excellent performance, with an average root mean square error (RMSE) lower than 0.27 m and a scaling accuracy higher than 96%, surpassing several baseline methodologies and competing with existing state-of-the-art reconstruction methods such as City3D and SimpliCity, while requiring far fewer visual data references. A comparison with LoD1 ground-truth models of the PLATEAU dataset demonstrates a 50% improvement in geometric proximities, confirming the robustness and adaptability of the framework. The involved artifacts are available at https://github.com/Alive59/ControlBldg/tree/master .