A Hybrid Deep Learning and Generative AI Framework for Knowledge‐Aware Evaluation of Handwritten Indic Scripts

ABSTRACT The automatic assessment of handwritten answers in low‐resource Indic scripts is a difficult problem because of the complexity of the orthographic ligatures, the variability of the writers, limited availability of annotated resources, and the error propagation from handwritten text recognition (HTR) to the grading modules. In this paper, we aim to address these challenges by introducing an end‐to‐end framework that combines Text Region Detection, sequential HTR, and Generative AI (GenAI) using Large Language Models (LLMs) for Explainable Multilingual Answer Assessment. The proposed framework was evaluated using a Telugu handwritten character dataset containing diverse character forms, including basic characters, vowel modifiers, compound characters, vattu combinations, and diacritics. A conditional Generative Adversarial Network (cGAN) was employed to generate synthetic Telugu character samples and increase handwriting diversity, which were integrated with the original training images. The framework was assessed using Character Recognition Accuracy (CRA), Word Recognition Accuracy (WRA), precision, recall, and F1‐score to evaluate recognition effectiveness and generalization. The proposed system carries out pre‐processing of document images and region awareness for Text Detection in document images using Character Region Awareness for Text Detection (CRAFT) based on non‐heuristic segmentation to avoid explicit character segmentation. Detected handwritten text regions are then recognized and transcribed to text by using a Convolutional Recurrent Neural Network with Connectionist Temporal Classification (CRNN‐CTC). An LLM‐based GenAI refinement module corrects accented characters, checks for word spelling errors, and corrects errors based on the context while maintaining the student's original meaning and intent and without spurious corrections. Next, with the text structured with the correct answer key, each is processed by a carefully designed text understanding module to locate the key concepts, to create a semantic map, and finally, using the rubric‐aware scoring from contextual embeddings, to identify answer points. The signals produce an explainable assessment tuple which includes awarded marks, concept coverage metrics, missing criteria and personalized LLM‐generated diagnostic feedback. The framework integrates computational intelligence through deep visual representation learning, generative augmentation, and knowledge‐aware semantic evaluation for robust recognition of complex handwritten Indic scripts. The proposed system achieves high transcription fidelity, strong linguistic refinement in multilingual Indic corpora of Hindi, Telugu, and Kannada, while also achieving high human–AI evaluation agreement, making it a scalable automated academic evaluation system.