MTEP: A Matched Temporal Evaluation Protocol for Cognitive Fatigue Detection from In-Task Self-Reports

Background/Objectives: Fatigue models are often trained on labels that participants produce, by an action, while the sensors record. We aimed to certify that an evaluation design’s estimates reflect the state rather than the report. Methods: We present MTEP, a matched temporal evaluation protocol delivering label and feature separation, participant-level generalisation, within-recording position balance and report-independence, meaning that no window used to predict a report overlaps the act producing it. We certify report-independence by lead displacement and position balance with linear and nonlinear models, calibrate both certificates with injected effects of known amplitude and shape, compare four evaluation designs on corpora of known truth, and apply MTEP to CogBeacon and CL-Drive. Results: On corpora of known truth, MTEP returned the correct verdict in 78 of 80 cases against 58–62 for designs in current use, which concluded that a detector works in 16–19 of 20 corpora containing only the reporting act (MTEP: 1 of 20). The report-independence certificate had a specificity of 39/40 and detected an act-borne component of 0.05 local standard deviations in 30/40 replicates; the nonlinear position certificate detected imbalances the linear test missed (10/10 vs. 0/10). On CogBeacon, MTEP certified at a lead of one round, with pooled ROC-AUC 0.659 carried by task performance. Our model exceeded every published single-classifier accuracy on the same EEG features and protocols, yet a sensor-free clock matched or exceeded published accuracies on both corpora. Conclusions: Report-independence is a distinct requirement that existing designs do not imply; it can be checked from the collection protocol and certified on the evaluation sample.