A leakage-controlled evaluation for meta-pretrained factorization machines in cold-start personalized learning

Cold-start educational recommendation must distinguish recovery of logged behavior from educational benefit. We evaluated a pooled factorization machine (Pooled FM) and a first-order meta-pretrained factorization machine (Meta-FM) on audited EdNet KT1 logs with 14,000 training, 3,000 development-validation, and 3,000 reserved test learners. Supports contain 1, 3, 5, or 10 complete bundle activities, and candidates comprise all 8,253 training-catalog bundles. Both methods share a sparse support-conditioned representation, matched initialization, a one-epoch training budget, and the same support-time evaluation update. The primary immediate deterministic endpoint is normalized discounted cumulative gain at rank 5 (nDCG@5), with three contrasts prespecified before held-out performance evaluation. Across five retained fitted seeds, held-out path nDCG@5 is 0.449031 ± 0.001990 for Pooled FM and 0.448597 ± 0.002068 for Meta-FM (mean ± sample standard deviation); the immediate bridge control is 0.445451. Meta-minus-Pooled is −0.000433969 [−0.000583808, −0.000289114], with a paired learner-cluster 95% interval. Pooled-minus-bridge is +0.003579978 [+0.002152788, +0.005165757] and Meta-minus-bridge is +0.003146009 [+0.001753661, +0.004723352]. These intervals condition on retained fitted seeds and the prespecified analysis; they do not quantify variability over new training runs. Earlier validation analyses show near-zero mean inference-update effects without proving equivalence and a material joint ordered-block/optimization effect under the matched budget. Prespecified stochastic validation and complementary Duolingo calibration remain separate endpoints; their held-out path utility is not established. The evidence supports careful horizon alignment and strong behavioral controls while avoiding general meta-training-superiority or causal learning-gain claims. No retraining, feature refitting, or test-driven selection occurred.

Authors

Institutions

Publication Details

Journal
Frontiers in Psychology
Published
2026-11-17
DOI
https://doi.org/10.3389/fpsyg.2026.1908866
Primary Topic
Recommender Systems and Techniques
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

A leakage-controlled evaluation for meta-pretrained factorization machines in cold-start personalized learning

Jiayi Wu, Yi Xiong
1 links
Frontiers in Psychology
Recommender Systems and Techniques
article

A leakage-controlled evaluation for meta-pretrained factorization machines in cold-start personalized learning

Jiayi Wu, Yi Xiong
article en

Abstract

Cold-start educational recommendation must distinguish recovery of logged behavior from educational benefit. We evaluated a pooled factorization machine (Pooled FM) and a first-order meta-pretrained factorization machine (Meta-FM) on audited EdNet KT1 logs with 14,000 training, 3,000 development-validation, and 3,000 reserved test learners. Supports contain 1, 3, 5, or 10 complete bundle activities, and candidates comprise all 8,253 training-catalog bundles. Both methods share a sparse support-conditioned representation, matched initialization, a one-epoch training budget, and the same support-time evaluation update. The primary immediate deterministic endpoint is normalized discounted cumulative gain at rank 5 (nDCG@5), with three contrasts prespecified before held-out performance evaluation. Across five retained fitted seeds, held-out path nDCG@5 is 0.449031 ± 0.001990 for Pooled FM and 0.448597 ± 0.002068 for Meta-FM (mean ± sample standard deviation); the immediate bridge control is 0.445451. Meta-minus-Pooled is −0.000433969 [−0.000583808, −0.000289114], with a paired learner-cluster 95% interval. Pooled-minus-bridge is +0.003579978 [+0.002152788, +0.005165757] and Meta-minus-bridge is +0.003146009 [+0.001753661, +0.004723352]. These intervals condition on retained fitted seeds and the prespecified analysis; they do not quantify variability over new training runs. Earlier validation analyses show near-zero mean inference-update effects without proving equivalence and a material joint ordered-block/optimization effect under the matched budget. Prespecified stochastic validation and complementary Duolingo calibration remain separate endpoints; their held-out path utility is not established. The evidence supports careful horizon alignment and strong behavioral controls while avoiding general meta-training-superiority or causal learning-gain claims. No retraining, feature refitting, or test-driven selection occurred.

Frontiers in PsychologyVol. 17
Hunan Railway Professional Technology College (CN), Hunan Vocational College of Railway Technology (CN), Hunan University of Technology (CN)
Recommender Systems and Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon