Learning from other cities: Transfer learning based multimodal residential energy prediction for cities with limited existing data

• Novel multimodal transfer learning application in residential energy prediction. • Integration of EPCs and Street view images for robust energy predictions. • Case studies across UK cities demonstrate adaptability of the proposed framework. • Proposed model enhances prediction accuracy in data-poor regions by up to 64 %. • XAI highlights floor and wall insulation as priorities for energy retrofits. Reliable prediction of residential energy consumption is essential for informing energy efficiency policies and retrofit strategies. However, traditional data-driven approaches are often constrained by the availability and quality of data. This study presents a novel approach combining multimodal neural networks with a transfer learning framework, leveraging both tabular and visual data to enhance prediction accuracy and enable knowledge transfer from data-rich to data-poor regions. Case studies conducted in Barnsley, Doncaster, and Merthyr Tydfil demonstrated that the proposed approach outperforms traditional mono-modal models. The multimodal model improved prediction accuracy significantly, achieving a MAPE reduction from 1.15 (with only visual data) and 0.86 (with only tabular data) to 0.43 (with both visual and tabular data), while the inclusion of transfer learning offers further performance improvements in data-scarce regions, with up to 63.6 % error reduction. Explainable AI is utilised to validate the model’s interpretability, confirming key features such as floor and wall insulation conditions as pivotal in energy consumption predictions. This integrated framework offers actionable insights for policymakers, facilitating data-driven decisions to enhance energy efficiency in diverse urban settings.