RoofSam: Adapting the Segment Anything Model to Rooftop Classification in Aerial Images

Foundation models, such as the Segment Anything Model (SAM), are transforming computer vision by enabling versatile architectures that can adapt to new tasks with minimal domain-specific data. In the geospatial context, however, these capabilities have been primarily examined for SAM's intended use case, image segmentation. This paper aims to extend SAM's capabilities beyond dense prediction by introducing RoofSAM, an adaptation of SAM specifically designed for roof shape classification in high-resolution aerial images guided by building outlines. RoofSAM utilizes SAM's interactive interface to effectively incorporate readily available prior knowledge about building locations through point-sampling processes. We evaluate several sampling strategies, including centroid and random sampling approaches, to determine their effectiveness in enhancing model accuracy. Our empirical and visual analyses demonstrate that RoofSAM can successfully transfer SAM's segmentation knowledge to the task of roof shape classification, achieving F1-scores of up to 86% for frequently represented classes, even with limited training data. These results underscore the versatility of vision foundation models in geospatial data processing applications.