Exploring multimodal deep learning for contextual operator classification in cartographic building generalization

Cartographic generalization has proven notoriously challenging to automate, owing to the difficulty of formalizing the implicit knowledge heavily employed throughout the process. Consequently, deep learning has emerged as a promising candidate for a paradigm shift in automated cartographic generalization, as it has the potential to circumvent explicit knowledge formalization by learning from examples. Current studies mainly revolve around ambitious end-to-end generalization approaches, deviating from established cartographic practices by largely dismissing the significance of generalization operators. In addition, they incorporate limited contextual information and predominantly process maps represented as rasters. Therefore, this thesis investigates the feasibility of using deep learning in conjunction with a novel, enriched dataset to predict contextual generalization operators (elimination, aggregation, typification, displacement, and enlargement) that are to be applied to generalize buildings on topographic maps during the transition from 1:25,000 to 1:50,000. To this end, classification models based on vector and raster representations are developed and evaluated. Furthermore, a multimodal model is proposed that exploits both modalities simultaneously to generate predictions. The study also explores the role of contextual map features in the form of surrounding buildings and roads in facilitating operator predictions. The results reveal that deep learning models can effectively predict cartographic generalization operators, particularly those less dependent on contextual information such as enlargement and aggregation. However, performance declines for highly contextual operators such as displacement and typification. The classification of elimination yields the worst evaluation metrics. Moreover, it is shown that the models are particularly adept at predicting operators for buildings located in rural areas. The incorporation of contextual map features is crucial, since their exclusion results in worse performance for certain operators. Across modalities, the multimodal model achieves the best overall classification evaluation metrics. These findings contribute to the advancement of automating cartographic generalization using deep learning, demonstrating its capacity to predict operators and underscoring the importance of contextual features and multimodal approaches. The study lays the groundwork for integrating such models into broader automated generalization workflows driven by deep learning. The code is available at github.com/jorissenn/genops.