Large Language Models for Building Energy Retrofit Decision-Making: Technical and Sociotechnical Evaluations

Conventional approaches to building energy retrofit decision-making struggle to generalize across diverse building characteristics, climate conditions, and occupant behaviors, and often lack interpretability. Generative AI, particularly Large Language Models (LLMs), offers a promising solution because they learn from extensive, heterogeneous data and can articulate inferences in transparent natural language. However, their capabilities in retrofit decision-making remain underexplored. This study evaluates six widely used LLMs on two objectives: determining the retrofit measure that maximizes CO2 reduction (a technical task) and minimizes the payback period (a sociotechnical task). We assessed performance across accuracy, consistency, sensitivity, and reasoning. The evaluation used 400 residential buildings from a nationwide, simulation-based database. The results reveal that LLMs vary across cases, with consistently strong technical-task performance but notably weaker performance on the sociotechnical one, highlighting limitations in handling complex economic and contextual trade-offs. The models consistently identify a near-optimal solution for the technical task (Top-5 accuracy reaching 92.8%), although their ability to pinpoint the single best option is limited (Top-1 accuracy reaching 54.5%). While models approximate engineering logic by prioritizing location and geometry, their reasoning processes are oversimplified. These findings suggest LLMs are promising for technical advisory tools but not yet reliable for standalone retrofit decision-making.