EnergyNet: A modality-aware attention fusion network for building energy efficiency classification

In the face of rising global energy demands, precise classification of building energy efficiency is critical for advancing sustainable energy practices. Traditional classification methods have been limited by their inability to effectively integrate diverse data types. Additionally, the valuable environmental information visible in building street view images has been consistently overlooked, leading to less comprehensive evaluations. This study introduces EnergyNet, an innovative framework designed to synergistically fuse multimodal data, including the environmental context that has previously been underutilized. The framework employs a state-of-the-art dual-branch architecture with a modality-aware attention mechanism to optimize the interpretation and fusion of both visual and textual data. Comparative experiments on real-world data demonstrate that EnergyNet substantially improves upon existing models, achieving an accuracy rate of 87.22% and an F1 score improvement of 5.39% over the best-performing benchmarks. The proven generalization capacity of the framework across different geographical regions highlights its potential as a scalable and effective solution for enhancing global energy efficiency measures. • Integrates visual and textual data for better building energy efficiency prediction. • Transfer learning extracts key features, enhancing predictions with limited data. • Modality-aware fusion combines features precisely with multi-head self-attention. • Outperforms traditional and state-of-the-art models in diverse real-world tests. • Performs consistently across regions, aiding energy conservation policies.