Qualitative Feedback Comparison Between Professors and AI in STEM Education

This study explores qualitative feedback provided by professors and ChatGPT 40 version when evaluating conclusions of laboratory reports. The experiment involves validating the final moisture content of shredded carrot through material balance calculations and thermogravimetric analysis. While reports are team-based, conclusions are individually written, resulting in 60 unique submissions focused on the argumentation of variations between estimated and measured moisture content. Thirdsemester students in the Bioengineering and Chemical Processes pathway at Tecnologico de Monterrey participated in this lab, part of the course “Application of the Principles of Conservation of Matter in Chemical and Biological Processes” (IQ1001B). Four professors established feedback guidelines, which were also provided to the AI. Advanced natural language processing (NLP) techniques were used to evaluate feedback, employing the Sentence-Transformer model ('all-MiniLM-L6-v2') to generate semantic embeddings and calculate cosine similarity. Statistical methods, including Shapiro-Wilk, paired t-tests, Wilcoxon signedrank, and Friedman tests, analyzed semantic alignment and significant differences. The study evaluates whether AI can provide feedback that aligns semantically with human input, focusing on the model's ability to identify meaning and context, emphasizing scalability and efficiency in large-scale STEM education, contributing to student-centered learning in 21st-century education.