LLM for Automated Answer Evaluation with DSPy Prompt-Fine-tuning

Generative AI and prompt engineering have the potential to create innovative applications in a short time. This paper explores the potential of generative AI and how prompt engineering can be used to build educational applications to help instructors automate question generation and answer evaluation in a short time. This paper details how optimized prompts are used in an educational application that was designed to support instructors and students by automating the evaluation of student answers to provide personalized feedback. To fine-tune the prompts, DSPy-enabled optimization was utilized to achieve consistency in LLM-generated questions, and answer evaluations. The paper details the implementation for the prompt-based question generation and creation of different variations of the of the question and answer evaluation using DSPy integrated method. The results demonstrate that the structured approach of DSPy to prompt generation significantly enhances the reliability of the system, ensuring that the LLM outputs are in line with the goals of the system, while maintaining consistency and interpretability.