Evaluating GPT-5 and Fine-Tuned Transformer Models for QUS-Based User Story Quality Verification

User stories are widely used in agile requirements engineering, but their informal, natural language-based structure can introduce ambiguity, incomplete information, excessive scope, and other flaws that affect subsequent development activities. This study investigates the automated verification of individual Quality User Story (QUS) criteria by comparing general-purpose generative models evaluated using prompts with specialized Transformer models using supervised fine-tuning. Specifically, GPT-5 and GPT-5 Thinking were evaluated under Context-Minimal Prompting (CMP) and Context-Rich Prompting (CRP), while four task-specialized single-label (SL) models—SL-BUSQ-v2, SL-RUSQ-v2, SL-DUSQ-v2, and SL-gemma-USQ—were trained using a two-stage adaptation strategy. In the first stage, the models were pre-adapted on a partial multi-label (ML) corpus of 1625 user stories to expose them to the coexistence of multiple QUS violations. In the second stage, the models were adjusted for independent binary classification tasks on the eight individual QUS criteria using a refined reference set of 960 stories. The results show that all specialized models achieved higher average F1-scores than the GPT-5 prompting configurations. SL-gemma-USQ obtained the highest average F1-score (0.957793), followed by SL-BUSQ-v2 (0.956753), while the strongest prompting-based baseline, GPT-5 CMP, reached 0.915080. Overall, the findings suggest that supervised task specialization is a promising strategy for QUS-based user story quality verification.