Explainable AI for education: Enhancing essay scoring via rubric-aligned chain-of-thought prompting

Document Type

Article

Publication Date

6-1-2026

Abstract

Automated essay scoring (AES) systems increasingly leverage fine-tuned large language models (LLMs) to enhance scoring accuracy and feedback generation. However, current LLM-based AES approaches often lack interpretability and consistent alignment with human scoring rubrics, limiting their practical adoption in educational settings. This study proposes QwenScore+, a novel framework that integrates rubric-aware Chain-of-Thought (CoT) prompting with reinforcement learning from human feedback (RLHF) to improve the transparency, quality and educational alignment of automated feedback. QwenScore+ is evaluated on a proprietary IELTS writing dataset comprising over 5000 essays annotated with trait-level scores and expert-written feedback. Experimental results demonstrate that QwenScore+ significantly outperforms strong baselines such as BERT, GPT-3.5 and GPT-4 in feedback generation, achieving higher BLEU, ROUGE-L and cosine similarity scores, alongside improvements in trait-level scoring measured by quadratic weighted kappa (QWK). Furthermore, rubric-aligned CoT prompting enables the generation of feedback that better mirrors human reasoning patterns, as confirmed through automatic metrics and human evaluations. These findings highlight the potential of combining explainable reasoning strategies with human-aligned reward optimization to develop more transparent, reliable and pedagogically valuable AES systems for real-world applications.

Keywords

Automated essay scoring (AES), chain-of-thought prompting, explainability, large language models (LLMs), rubric alignment

Publication Title

International Journal of Modern Physics C

ISSN

0129-1831

DOI

10.1142/S0129183125420136

Volume

37

Issue

6

Share

COinS