Unleashing the Power of Generative AI and LLM for Training Evaluation

Abstract This paper presents a novel approach to training evaluation using Language Models (LLM) and Generative AI (GenAI) to build a classification model. The study aims to develop a resource-efficient solution for analyzing training rubrics and transcripts, thereby enhancing the assessment of learning outcomes and performance. The methodology involves data collection, preprocessing, model fine-tuning, prompting, and evaluation. A pre-trained LLM model is fine-tuned on preprocessed training rubrics and transcripts, allowing it to adapt to specific language patterns and structures. The model generates prompts to classify training materials based on predefined criteria, with domain expertise incorporated for complex evaluation rules. Results demonstrate a 60% reduction in processing time for evaluating rubrics and transcripts compared to manual assessment. The implemented model has significantly reduced the workload of the training department and improved the efficiency of data analysis. Furthermore, the model's feedback has led to targeted improvements in training content, resulting in higher learner satisfaction. This innovative application of LLM and GenAI in training evaluation offers a new perspective on leveraging AI to enhance educational processes and outcomes in a resource-efficient manner.