How well do large language models perform in green building assessment? An evaluation analysis from a case data set

Recent advances in Large Language Models (LLMs) have demonstrated their impressive capabilities in various tasks. However, their potential in the specialized field of green building assessment has not been explored. Such a study is necessary to understand their performance in this domain, with the goal of optimizing LLMs to reduce the workload of manual green building assessments and enable designers to conduct preliminary self-assessments more efficiently and economically. In this regard, this study expands the dataset from 112 to 1200 real-world cases, and then eleven leading LLMs are selected for evaluation using both long and short text inputs combined with three different prompt engineering techniques (i.e., zer0-shot, zero-shot CoT, few-shot) to determine their accuracy. The findings indicate that LLMs perform better with short text inputs, particularly GPT-4, which showed the highest effectiveness in the green building evaluation field. Prompt engineering improved the performance of GPT-4 with short text inputs, though its effectiveness varied across different LLMs. Furthermore, LLMs excelled in evaluating qualitative criteria that do not require logical reasoning but performed poorly in assessing quantitative criteria that involve complex mathematical calculations. Research findings provide valuable insights for future development of LLM-based methods for green building evaluation, aiming to alleviate current manual assessment burdens and improve design review processes.