An interpretable framework for task response in IELTS writing: integrating thematic analysis, argumentation dimensions, and GPT-4 for educational feedback

The assessment of Task Response (TR) in high-stakes exams like IELTS is plagued by scorer subjectivity, conceptual vagueness, and limited pedagogical utility, which Automated Writing Evaluation (AWE) tools, including “black-box” Generative AI, have not consistently addressed in formative contexts. This challenge is further compounded by variable levels of feedback literacy among learners. To bridge these gaps, this exploratory, proof-of-concept study develops an interpretable, five-dimensional TR framework (Coverage, Opinion, Claim, Grounds, Penalty) through thematic analysis of 28 official IELTS essays (Bands 5.0–7.0). The proposed framework demonstrates promising inter-rater reliability (Cohen’s kappa = 0.827) and strong internal content validity (S-CVI = 0.92) within the context of official descriptors. Preliminary results from Kruskal-Wallis (p < 0.001) and Spearman correlations (r > 0.90 for Claim and Grounds) suggest its potential to discriminate between proficiency levels. By mapping this framework onto GPT-4 via optimized prompts, the study shifts AI’s role from an opaque scorer (Mean Absolute Error, MAE = 0.54; correlation with human scores, r = 0.39) toward a structured diagnostic feedback tool (MAE = 0.41; r = 0.61). While these results represent a promising proof-of-concept for formative use, the levels of agreement do not yet meet the requirements for high-stakes summative assessment. Given the limited sample size and exploratory design, these findings should be interpreted cautiously as preliminary evidence for framework viability. The study highlights the framework’s potential to enhance pedagogical transparency and support self-regulation through ZPD-aligned interventions. Future research is needed to validate the framework in larger and more diverse samples and to examine its practical impact in classroom settings.