From Black-Box Automated Scoring to Engaging Feedback: A Docimological-Pedagogical Pipeline for Assessment with Local LLMs
The use of artificial intelligence in assessment raises questions about how automated judgements relate to learning goals and how students can use the feedback they receive. This paper presents the first stage of a broader research project on the assessment of written responses in Italian provided by local large language models (LLMs). We conducted a purposive narrative review of ten technical publications, retrieved through ACL Anthology, arXiv and publishers’ websites before1 October 2026, utilizing literature on feedback and assessment practices. The synthesis shows that scoring performance depends on the task, the human reference and the evaluation setting, while evidence concerning feedback use remains limited. These findings inform a proposed five-layer pipeline in which explicit criteria and independently rated responses provide the basis for semantic analysis, local-model evaluations and teacher review. The pedagogical rationale is to support judgements whose grounds can be verified and feedback that learners can use in their revision. As the pipeline has not yet been tested, we outline a validation program addressing scoring agreement, the quality of reviewed decisions, teacher workload and students’ interpretation and use of feedback.
Authors
- Marco Giganti
- Marco Lazzari (ORCID: https://orcid.org/0000-0003-1961-9914)
Publication Details
- Journal
- Aisberg (University of Bergamo)
- Published
- 2026-11-20
- Primary Topic
- Student Assessment and Feedback
- Type
- article
- Field-Weighted Citation Impact
- 0.00