A Pilot Study on ChatGPT’s Potential in Developing Construction Education Assessments Through a Lens of Bloom’s Taxonomy

Effective and quality assessment is crucial in construction management (CM) education to evaluate students’ understanding of the diverse skills demanded by the industry. Integrated by the American Council for Construction Education (ACCE) guidelines for its accredited CM programs, Bloom's Taxonomy provides a systematic and structured framework for aligning learning objectives with cognitive processes, guiding educators in developing assessments. However, under real-world practice, crafting quality assessment questions that align with Bloom's Taxonomy and the ACCE student learning outcomes (SLOs) poses challenges for CM instructors due to the specialized expertise and complexity of knowledge. This study investigates the potential of ChatGPT, an LLM-based chatbot, to assist educators in generating assessment questions specific to construction education with the integration of Bloom's Taxonomy. Focusing on two specific SLOs: 1) create construction project cost estimates, and 2) analyze methods, materials, and equipment to construct projects. The study adopted the latest ChatGPT o1-preview model for question generation (QG). These questions were then evaluated by the researchers based on criteria and alignment with Bloom's Taxonomy, clarity, sufficiency of information, feasibility, and relevance to learning objectives. Findings revealed that while ChatGPT can produce relevant and cognitively appropriate questions related to the two SLOs, several limitations still exist regarding clarity and completeness of information. The study concludes that ChatGPT is a potent tool to support CM educators in developing assessment materials but requires proper prompt engineering and additional context to enhance question quality. Future research should involve more data collection methods and a tailored database for ChatGPT training.