Drone imagery for roof detection, classification, and segmentation to support mosquito-borne disease risk assessment: The Nacala-Roof-Material dataset

Since low-quality housing and in particular certain roof characteristics are associated with an increased risk of malaria, classification of roof types based on remote sensing imagery can aid in assessing malaria risk and thereby help prevent the disease. To support research in this area, we release the Nacala-Roof-Material dataset 1 1 The data and code are available through our project page: https://mosquito-risk.github.io/Nacala . The data can be explored interactively using this application: https://mosquito-risk.github.io/Nacala/demo . , which contains high-resolution drone images from Mozambique with corresponding labels delineating houses and specifying their roof types. The dataset defines a multi-task computer vision problem, encompassing object detection, classification, and segmentation. Additionally, we benchmarked various state-of-the-art approaches on the dataset. Canonical U-Nets, YOLOv8, and a custom decoder on pretrained DINOv2 served as baselines. We demonstrate that while each method has its advantages, none is superior across all metrics, highlighting the potential of our dataset for future research in multi-task learning. Although the tasks are closely related, accurate segmentation of objects does not necessarily imply accurate instance separation, and vice versa. We address this general issue by introducing a variant of the deep ordinal watershed (DOW) approach, which additionally segments the interior of objects, allowing for improved object delineation and separation. We show that our DOW variant is a versatile approach that improves the performance of both U-Net and DINOv2 backbones, mitigating potential conflicts between semantic seVenkanna Guthulagmentation and instance segmentation.