Enabling GPTs\nfor Expert-Level Environmental Engineering\nQuestion Answering

Artificial intelligence (AI) holds\nsignificant potential\nfor advancing\nresearch and development in the field of environmental science and\nengineering (ESE), but the development of domain-specific large language\nmodels (LLMs) in this field has not been reported. This study addresses\nthis gap by evaluating the performance of advanced LLMs in answering\nexpert-level, closed-book environmental engineering questions. We\nassessed two generative pretrained transformer (GPT) models and five\nfine-tuned models (FTMs) on an expert-level question answering (QA)\ndata set, focusing on relevance (from 0 to 1), factuality (0 to 1),\nformat, richness, QA difficulty level, and domain topic. Results show\nthat GPT-4 achieves a relevance score of 0.644 and a factuality score\nof 0.791 based on 286 questions, indicating room for improvement,\nparticularly for more difficult questions (scores dropped to below\n0.5). Notably, FTMs with larger data sets resisted factuality degradation,\nhighlighting the need for high-quality training materials. Inaccuracies\nand format issues are often linked to overtraining and catastrophic\ninterference. This first investigation leverages expert-level textbooks\nto enhance LLM performance, thereby providing valuable insights and\nsetting the stage for developing more robust domain-specific LLMs\nfor environmental applications.