Structured human resource development with AI support is associated with higher expert rated recruitment task quality among education graduates

Abstract Graduate employability programmes are usually evaluated with self-report measures. We compared a structured human resource development programme supported by generative AI with career guidance matched on topics, duration and facilitator contact. Twelve intact cohorts of graduates seeking education-sector posts in Guangdong, China (N = 186), were allocated as whole units, six per condition, crossed with institution, facilitator and schedule. Blinded experts gave a global rating (0–100) of recruitment-task materials produced on site, under invigilation and without AI, before and after the programme. Ratings were made without a pre-specified rubric, which limits their interpretation. Among 168 participants with post-test data (overall attrition 9.7%), the adjusted difference favoured the AI-supported programme by 5.38 points (95% CI 2.76 to 8.01; d = 0.60) and was stable across eight specifications. Four new self-report scales showed smaller differences (d = 0.12 to 0.29), none significant after Holm correction; each scale could detect only differences of about d = 0.4. The effect on expert ratings exceeded the mean effect on self-report by 0.37 standard deviations (95% CI 0.10 to 0.63). Component analyses were exploratory. The design cannot separate the contribution of AI from structured practice and feedback.