Cross-cohort validation of EEG biomarkers using a pre-registered three-criterion framework: a demonstration in amyotrophic lateral sclerosis

Background. Despite decades of EEG biomarker development for amyotrophic lateral sclerosis (ALS), no candidate has progressed to clinical deployment, partly because cross-cohort validation is rare. We aimed to develop and demonstrate a pre-registered three-criterion screening framework to identify which candidate EEG features show the combined properties required for biomarker translation: cross-sectional discrimination, within-subject reliability, and cross-cohort effect consistency. Methods. We pre-registered a validation framework adapted from FDA-NIH BEST principles and applied it to 18 EEG features across three independent cohorts: EEGET-ALS (six ALS patients, 170 healthy controls; home recordings over 17 days), UCL/Penn State (eight ALS patients; clinic recordings over 1-2 months), and a healthy adult test-retest reference (60 participants, three sessions over one month). Features were required to meet three pre-registered criteria: (1) cross-sectional discrimination (Cohen's d > 0.5), (2) within-subject reliability (ICC(3,1) > 0.6 in both ALS cohorts independently), and (3) cross-cohort effect direction consistency. Cross-cohort permutation testing (1000 iterations) and subject-level bootstrap resampling (500 iterations) were applied to assess robustness. Results. Three features met all three pre-registered criteria: gamma power, theta power, and spectral entropy. Cross-cohort permutation testing showed that no permutation, out of 1000, produced any feature in the three-cohort reliability intersection (Monte Carlo p = 0.001). Bootstrap resampling indicated that the composition of the selected set was sensitive to sample composition (median count 1, 95% CI [0, 4]), with gamma power (67% inclusion), spectral entropy (43%), and theta power (32%) the most stable features at a relaxed reliability threshold (ICC > 0.5); at the primary pre-registered threshold (ICC > 0.6), the corresponding inclusion probabilities are lower (57%, 27%, and 19%, respectively). A principal methodological finding was that cross-cohort differences in feature reliability were large relative to the apparent disease-associated signal, most clearly demonstrated by relative beta power (ICC = 0.09 in a home-recorded cohort versus 0.93 in a clinic-recorded cohort); because recording protocol, hardware, setting, population, and preprocessing all differ jointly between cohorts in this dataset, this is evidence of cohort-associated variability that plausibly includes a protocol contribution, not a separately identified, isolated protocol effect. Conclusions. This study provides a pre-registered, multi-cohort EEG biomarker screening framework for ALS with explicit bootstrap-quantified uncertainty. Gamma power, spectral entropy, and theta power are identified as priority candidates for prospective validation. A recurring methodological observation is that protocol-related cohort differences appear to contribute more to cross-cohort variability than disease-related effects within this dataset; because protocol is confounded with cohort, equipment, and preprocessing, this pattern is descriptive rather than causal. The observation nonetheless has direct implications for the design of multi-site EEG biomarker studies in neural engineering.

Authors

Institutions

Publication Details

Journal
medRxiv
Published
2026-11-12
DOI
https://doi.org/10.64898/2026.11.10.26367741
Primary Topic
Amyotrophic Lateral Sclerosis Research
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Cross-cohort validation of EEG biomarkers using a pre-registered three-criterion framework: a demonstration in amyotrophic lateral sclerosis

Haben Yhdego, Tewolde Teklu, Belay Gebregergis
medRxiv
Amyotrophic Lateral Sclerosis Research
preprint

Cross-cohort validation of EEG biomarkers using a pre-registered three-criterion framework: a demonstration in amyotrophic lateral sclerosis

Haben Yhdego, Tewolde Teklu, Belay Gebregergis
preprint en

Abstract

Background. Despite decades of EEG biomarker development for amyotrophic lateral sclerosis (ALS), no candidate has progressed to clinical deployment, partly because cross-cohort validation is rare. We aimed to develop and demonstrate a pre-registered three-criterion screening framework to identify which candidate EEG features show the combined properties required for biomarker translation: cross-sectional discrimination, within-subject reliability, and cross-cohort effect consistency. Methods. We pre-registered a validation framework adapted from FDA-NIH BEST principles and applied it to 18 EEG features across three independent cohorts: EEGET-ALS (six ALS patients, 170 healthy controls; home recordings over 17 days), UCL/Penn State (eight ALS patients; clinic recordings over 1-2 months), and a healthy adult test-retest reference (60 participants, three sessions over one month). Features were required to meet three pre-registered criteria: (1) cross-sectional discrimination (Cohen's d > 0.5), (2) within-subject reliability (ICC(3,1) > 0.6 in both ALS cohorts independently), and (3) cross-cohort effect direction consistency. Cross-cohort permutation testing (1000 iterations) and subject-level bootstrap resampling (500 iterations) were applied to assess robustness. Results. Three features met all three pre-registered criteria: gamma power, theta power, and spectral entropy. Cross-cohort permutation testing showed that no permutation, out of 1000, produced any feature in the three-cohort reliability intersection (Monte Carlo p = 0.001). Bootstrap resampling indicated that the composition of the selected set was sensitive to sample composition (median count 1, 95% CI [0, 4]), with gamma power (67% inclusion), spectral entropy (43%), and theta power (32%) the most stable features at a relaxed reliability threshold (ICC > 0.5); at the primary pre-registered threshold (ICC > 0.6), the corresponding inclusion probabilities are lower (57%, 27%, and 19%, respectively). A principal methodological finding was that cross-cohort differences in feature reliability were large relative to the apparent disease-associated signal, most clearly demonstrated by relative beta power (ICC = 0.09 in a home-recorded cohort versus 0.93 in a clinic-recorded cohort); because recording protocol, hardware, setting, population, and preprocessing all differ jointly between cohorts in this dataset, this is evidence of cohort-associated variability that plausibly includes a protocol contribution, not a separately identified, isolated protocol effect. Conclusions. This study provides a pre-registered, multi-cohort EEG biomarker screening framework for ALS with explicit bootstrap-quantified uncertainty. Gamma power, spectral entropy, and theta power are identified as priority candidates for prospective validation. A recurring methodological observation is that protocol-related cohort differences appear to contribute more to cross-cohort variability than disease-related effects within this dataset; because protocol is confounded with cohort, equipment, and preprocessing, this pattern is descriptive rather than causal. The observation nonetheless has direct implications for the design of multi-site EEG biomarker studies in neural engineering.

medRxiv
University of Toronto (CA), Old Dominion University (US)
Good health and well-being
Amyotrophic Lateral Sclerosis Research
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon