A leakage-controlled evaluation for meta-pretrained factorization machines in cold-start personalized learning
Cold-start educational recommendation must distinguish recovery of logged behavior from educational benefit. We evaluated a pooled factorization machine (Pooled FM) and a first-order meta-pretrained factorization machine (Meta-FM) on audited EdNet KT1 logs with 14,000 training, 3,000 development-validation, and 3,000 reserved test learners. Supports contain 1, 3, 5, or 10 complete bundle activities, and candidates comprise all 8,253 training-catalog bundles. Both methods share a sparse support-conditioned representation, matched initialization, a one-epoch training budget, and the same support-time evaluation update. The primary immediate deterministic endpoint is normalized discounted cumulative gain at rank 5 (nDCG@5), with three contrasts prespecified before held-out performance evaluation. Across five retained fitted seeds, held-out path nDCG@5 is 0.449031 ± 0.001990 for Pooled FM and 0.448597 ± 0.002068 for Meta-FM (mean ± sample standard deviation); the immediate bridge control is 0.445451. Meta-minus-Pooled is −0.000433969 [−0.000583808, −0.000289114], with a paired learner-cluster 95% interval. Pooled-minus-bridge is +0.003579978 [+0.002152788, +0.005165757] and Meta-minus-bridge is +0.003146009 [+0.001753661, +0.004723352]. These intervals condition on retained fitted seeds and the prespecified analysis; they do not quantify variability over new training runs. Earlier validation analyses show near-zero mean inference-update effects without proving equivalence and a material joint ordered-block/optimization effect under the matched budget. Prespecified stochastic validation and complementary Duolingo calibration remain separate endpoints; their held-out path utility is not established. The evidence supports careful horizon alignment and strong behavioral controls while avoiding general meta-training-superiority or causal learning-gain claims. No retraining, feature refitting, or test-driven selection occurred.
Authors
- Jiayi Wu
- Yi Xiong
Institutions
- Hunan Railway Professional Technology College (CN)
- Hunan Vocational College of Railway Technology (CN)
- Hunan University of Technology (CN)
Publication Details
- Journal
- Frontiers in Psychology
- Published
- 2026-11-17
- DOI
- https://doi.org/10.3389/fpsyg.2026.1908866
- Primary Topic
- Recommender Systems and Techniques
- Type
- article
- Field-Weighted Citation Impact
- 0.00