MAFNet: A multi-modal adaptive fusion network-based approach for individual building extraction from oblique photogrammetry

Oblique photogrammetry is a pivotal technique for generating high-precision 3D urban models. However, the resulting models often exhibit a shell-like, surface-only structure, lacking both physical separation and semantic distinction between individual buildings. This limitation restricts their capacity to support querying, analysis, and attribute management in digital city applications. To address the shortcomings of existing methods in accuracy, efficiency, and automation, this paper proposes a Multi-modal Adaptive Fusion Network (MAFNet). By employing a dual-branch encoder and an innovative Dual-modal Dynamic Feature Fusion (DDFF) module, MAFNet enables effective cross-modal feature interaction, which yields marked improvements in segmentation accuracy and robustness for complex urban scenes. On top of MAFNet, we further design a systematic automated processing workflow that associates the 2D segmentation results with the oblique photogrammetry model via edge detection, contour extraction and simplification, geographic coordinate mapping, and 3D visualization. This framework thus establishes a complete 2D-to-3D monolithization pipeline. To validate the proposed method, we construct an oblique photogrammetry building dataset including 3D models, DOM, DSM, and pixel-level mask annotations. Experimental results show that MAFNet achieves an IoU of 90.75% and an F1 score of 95.08% on this dataset, outperforming state-of-the-art models, thereby demonstrating the method’s efficacy and its strong potential for practical deployment in 3D GIS and digital twin systems. • Propose MAFNet, a dual-branch CNN with a fusion module for building extraction. • Design a workflow for automated building monolithization and 3D visualization. • Construct a novel high-resolution oblique photogrammetric building dataset. • Achieve superior performance in building contour extraction and 3D visualization.