Remote Sensing for Building Energy Efficiency Assessment: A Multi-modal Multi-task Approach

Buildings account for a substantial portion of global energy consumption, underscoring the importance of accurately assessing their energy efficiency. Traditional approaches for evaluating building energy efficiency typically rely on extensive tabular data gathered through labor-intensive on-site surveys, impeding their practicality for large-scale applications. Remote sensing data has emerged as a promising alternative due to its extensive spatial coverage and growing availability. However, existing approaches that utilize single-modality remote sensing data are limited in their capacity of predicting building energy efficiency. In this study, we propose a novel multi-modal, multitask neural network to model building energy efficiency using remote sensing sources. Our approach builds upon three key pillars: the integration of diverse remote sensing data, the implicit inclusion of tabular attributes during training and the application of foundation models for feature extraction. Extensive experiments on data from the city of Peterborough demonstrate the effectiveness of our method, achieving an F1 score of 68.14% for energy-efficiency classification. These results highlight our approach’s potential to enhance building energy modeling, providing a scalable and efficient alternative to traditional energy assessment techniques.