LLM-based Maritime Training Feedback System: Implementing RAG-Enhanced Assessment Analysis with STCW Compliance

This paper presents the implementation and evaluation of a Retrieval-Augmented Generation (RAG) system designed to provide automatic STCW- compliant feedback on maritime assessment questions. Building on preliminary findings from ongoing research into technological proficiency [1] (β=0.457) and institutional readiness [2] (β=0.341), this implementation addresses a critical gap: the need for automated feedback systems that maintain regulatory alignment while reducing instructor workload. The system utilizes the Mistral-7B large language model optimized with QLoRA for efficient local deployment, combined with a RAG architecture to ensure contextually relevant feedback. Evaluation results demonstrate the system’s ability to generate accurate feedback with response times under 15 seconds and STCW concept coverage of 85%, addressing key implementation barriers identified in our previous studies. The paper discusses how this implementation addresses technological proficiency barriers (β=0.457, p<0.001) and enhances perceived usefulness through automated, standards-compliant feedback that supports both individual competency development and institutional readiness.