A Multilingual Dataset of Student Answers, Human Grading, and Multi-LLM Evaluations for Automated Assessment Research Using JorGPT

The increasing adoption of Large Language Models (LLMs) in higher education has created a need for high-quality, publicly available benchmarks for automated assessment. Existing datasets often rely on synthetic responses or lack detailed human feedback. This paper presents a multilingual dataset of 3041 authentic student answers to 50 open-ended Computer Science questions, collected from real university assessments during the 2025–2026 academic year. The dataset includes the original student responses (Spanish) and their parallel translations (English), instructor (or teacher) defined ideal answers, blind human grading with qualitative feedback, and structured evaluations from three state-of-the-art LLMs (DeepSeek-chat-V3.2, Qwen-flash-2025-07-28, Gemini-2.5-flash-lite-001) using a unified JSON schema. This resource enables reproducible research in automated grading, feedback generation, and cross-lingual educational NLP.

Authors

Institutions

Publication Details

Journal
Data
Published
2026-03-17
DOI
https://doi.org/10.3390/data11030059
Primary Topic
Intelligent Tutoring Systems and Adaptive Learning
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

A Multilingual Dataset of Student Answers, Human Grading, and Multi-LLM Evaluations for Automated Assessment Research Using JorGPT

Javier Sánchez-Soriano, Yolanda Cerezo, Iván Barcia-Santos, Jorge Cisneros-González et al.
Data
Intelligent Tutoring Systems and Adaptive Learning
article

A Multilingual Dataset of Student Answers, Human Grading, and Multi-LLM Evaluations for Automated Assessment Research Using JorGPT

Javier Sánchez-Soriano, Yolanda Cerezo, Iván Barcia-Santos, Jorge Cisneros-González, Natalia Gordo-Herrera
article en

Abstract

The increasing adoption of Large Language Models (LLMs) in higher education has created a need for high-quality, publicly available benchmarks for automated assessment. Existing datasets often rely on synthetic responses or lack detailed human feedback. This paper presents a multilingual dataset of 3041 authentic student answers to 50 open-ended Computer Science questions, collected from real university assessments during the 2025–2026 academic year. The dataset includes the original student responses (Spanish) and their parallel translations (English), instructor (or teacher) defined ideal answers, blind human grading with qualitative feedback, and structured evaluations from three state-of-the-art LLMs (DeepSeek-chat-V3.2, Qwen-flash-2025-07-28, Gemini-2.5-flash-lite-001) using a unified JSON schema. This resource enables reproducible research in automated grading, feedback generation, and cross-lingual educational NLP.

DataVol. 11(3)
Universidad Francisco de Vitoria (ES)
Quality Education
Openalex Percentile: Top 22%
Intelligent Tutoring Systems and Adaptive Learning
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.