RUBRA: An Agentic AI System for Automatic Short Answer Grading Using LLMs and RAG

The proliferation of deep Learning applications in natural language processing has facilitated automated evaluation of short-answer questions, providing more transparent, interpretable and unbiased evaluation of students’ responses. In this work, we introduce RUBRA, an agentic AI system for Automatic Short Answer Grading (ASAG) using Large Language Models (LLM) and Retrieval-Augmented Generation (RAG) techniques. The system incorporates domain-specific context information retrieved from textbooks and study materials through a RAG framework and a dynamically generated rubric from reference answers provided by human tutors. Then, by aligning student responses with the rubric via semantic similarity scoring, the system computes an explainable and fair grade and provides transparent feedback to the learner. We discuss the architecture, components, and potential effectiveness of the proposed agentic AI system and tested our results over the SCIENTSBANK dataset. It is observed that the proposed approach not only outperformed the existing approaches in prediction accuracy but also significantly improved the autonomy and adaptability of the system.