Computational text indicators of epistemic norms in higher education: auditable cross-validated prediction against a weak observed household-resource baseline
Purpose This study tests whether computational text indicators of epistemic norms add out-of-sample predictive value for normalized academic results beyond a weak baseline of directly observed household/resource indicators. Design/methodology/approach Data comprise 263 students observed in two waves across five sites. Prediction uses 263 follow-up records. The baseline contains seven observed indicators. Archived epistemic counts were generated through pre-assigned mappings for closed items and LDA/topic-model-assisted processing for open responses. A deterministic 10-fold split (seed 60126) evaluated raw counts, within-fold PCA, and fixed outcome-independent ENA coordinates using pooled OOF R2, RMSE and MAE. Findings No epistemic specification reliably improved the baseline. Pooled OOF R2 was −0.009 (M0), −0.065 (M1), −0.032 (M1-PCA) and −0.008 (M2). M1 significantly worsened MAE after Holm correction; H1 and H2 were not supported. Research limitations/implications The baseline is weak, no prior-achievement variable is available, corpus-level reliability of automated epistemic variables is unknown, ENA coordinates were fixed before resampling, and program representation is imbalanced. Practical implications The indicators should not be used for high-stakes decisions without prospective measurement, subgroup-fairness and instructional-utility validation. Originality/value The study provides an auditable reporting object linking feature provenance, leakage risk, resampling controls and reporting requirements, and shows why null or adverse predictive results should be retained.
Authors
- Carlos Alberto Echeverría Mayorga (ORCID: https://orcid.org/0000-0002-4261-7178)
- Eduardo Antonio Bautista Sánchez (ORCID: https://orcid.org/0009-0003-7032-203X)
- José Miguel Esperanza
Institutions
- Universidad Salvadoreña Alberto Masferrer (SV)
Publication Details
- Journal
- Journal of Applied Research in Higher Education
- Published
- 2026-11-02
- DOI
- https://doi.org/10.1108/jarhe-03-2026-0387
- Primary Topic
- Online Learning and Analytics
- Type
- article
- Field-Weighted Citation Impact
- 0.00