Pixels, points and polygons: A dataset and benchmark for multimodal building vectorization

Accurate and up-to-date building polygon maps are essential for urban planning, disaster response, and large-scale geospatial analysis, yet creating them automatically across diverse regions remains challenging. To address this challenge, we present the P 3 dataset, a large-scale multimodal benchmark for building vectorization, constructed from aerial LiDAR point clouds, aerial images, and vectorized 2D building outlines, collected across three continents. The dataset contains over 10 billion LiDAR points with decimeter-level accuracy and RGB images at a ground sampling distance of 25 centimeters. While many existing datasets primarily focus on the image modality, P 3 offers a complementary perspective by also incorporating dense 3D information. We demonstrate that LiDAR point clouds serve as a robust modality for predicting building polygons, both in hybrid and end-to-end learning frameworks. Moreover, fusing aerial LiDAR and imagery further improves accuracy and geometric quality of predicted polygons. The P 3 dataset is publicly available, along with code and pretrained weights of three state-of-the-art models for building polygon prediction at https://github.com/raphaelsulzer/PixelsPointsPolygons .