Computational text indicators of epistemic norms in higher education: auditable cross-validated prediction against a weak observed household-resource baseline

Purpose This study tests whether computational text indicators of epistemic norms add out-of-sample predictive value for normalized academic results beyond a weak baseline of directly observed household/resource indicators. Design/methodology/approach Data comprise 263 students observed in two waves across five sites. Prediction uses 263 follow-up records. The baseline contains seven observed indicators. Archived epistemic counts were generated through pre-assigned mappings for closed items and LDA/topic-model-assisted processing for open responses. A deterministic 10-fold split (seed 60126) evaluated raw counts, within-fold PCA, and fixed outcome-independent ENA coordinates using pooled OOF R2, RMSE and MAE. Findings No epistemic specification reliably improved the baseline. Pooled OOF R2 was −0.009 (M0), −0.065 (M1), −0.032 (M1-PCA) and −0.008 (M2). M1 significantly worsened MAE after Holm correction; H1 and H2 were not supported. Research limitations/implications The baseline is weak, no prior-achievement variable is available, corpus-level reliability of automated epistemic variables is unknown, ENA coordinates were fixed before resampling, and program representation is imbalanced. Practical implications The indicators should not be used for high-stakes decisions without prospective measurement, subgroup-fairness and instructional-utility validation. Originality/value The study provides an auditable reporting object linking feature provenance, leakage risk, resampling controls and reporting requirements, and shows why null or adverse predictive results should be retained.

Authors

Institutions

Publication Details

Journal
Journal of Applied Research in Higher Education
Published
2026-11-02
DOI
https://doi.org/10.1108/jarhe-03-2026-0387
Primary Topic
Online Learning and Analytics
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Computational text indicators of epistemic norms in higher education: auditable cross-validated prediction against a weak observed household-resource baseline

Carlos Alberto Echeverría Mayorga, Eduardo Antonio Bautista Sánchez, José Miguel Esperanza
1 links
Journal of Applied Research in Higher Education
Online Learning and Analytics
article

Computational text indicators of epistemic norms in higher education: auditable cross-validated prediction against a weak observed household-resource baseline

Carlos Alberto Echeverría Mayorga, Eduardo Antonio Bautista Sánchez, José Miguel Esperanza
article en

Abstract

Purpose This study tests whether computational text indicators of epistemic norms add out-of-sample predictive value for normalized academic results beyond a weak baseline of directly observed household/resource indicators. Design/methodology/approach Data comprise 263 students observed in two waves across five sites. Prediction uses 263 follow-up records. The baseline contains seven observed indicators. Archived epistemic counts were generated through pre-assigned mappings for closed items and LDA/topic-model-assisted processing for open responses. A deterministic 10-fold split (seed 60126) evaluated raw counts, within-fold PCA, and fixed outcome-independent ENA coordinates using pooled OOF R2, RMSE and MAE. Findings No epistemic specification reliably improved the baseline. Pooled OOF R2 was −0.009 (M0), −0.065 (M1), −0.032 (M1-PCA) and −0.008 (M2). M1 significantly worsened MAE after Holm correction; H1 and H2 were not supported. Research limitations/implications The baseline is weak, no prior-achievement variable is available, corpus-level reliability of automated epistemic variables is unknown, ENA coordinates were fixed before resampling, and program representation is imbalanced. Practical implications The indicators should not be used for high-stakes decisions without prospective measurement, subgroup-fairness and instructional-utility validation. Originality/value The study provides an auditable reporting object linking feature provenance, leakage risk, resampling controls and reporting requirements, and shows why null or adverse predictive results should be retained.

Journal of Applied Research in Higher Education
Universidad Salvadoreña Alberto Masferrer (SV)
Quality education
Openalex Percentile: Top 11%
Online Learning and Analytics
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon
Computational text indicators of epistemic norms in higher education: auditable cross-validated prediction against a weak observed household-resource baseline — Carlos Alberto Echeverría Mayorga, Eduardo Antonio Bautista Sánchez, et al. · Journal of Applied Research in Higher Education (2026) | TGRS Research Map | TGRS