Disrupted near-critical dynamics and fragmented sleep in insomnia: evidence from neural avalanche analysis of EEG

<i>Objective.</i>Sleep is hypothesized to restore near-critical dynamics in large-scale brain networks, whereas insomnia may disrupt this self-organizing process. This study aimed to determine whether insomnia alters neural avalanche dynamics and criticality-based EEG metrics, and whether these metrics enhance prediction of sleep fragmentation compared with conventional spectral measures.<i>Approach.</i>Overnight high-density electroencephalography was recorded from 50 participants aged 16-69 years, including healthy sleepers and individuals with insomnia. Neural avalanches were detected as clusters of significant amplitude excursions. The branching parameter (<i>σ</i>) quantified temporal propagation within avalanches, while the deviation-from-criticality coefficient (DCC) indexed the system's distance from the critical state. These criticality features were contrasted with spectral power measures in predictive models of non-rapid eye movement (NREM) sleep fragmentation.<i>Main results.</i>Participants with insomnia exhibited reduced avalanche density and diminished slow-wave activity, accompanied by significant deviations of<i>σ</i>from the critical value and elevated DCC across the night. Criticality-based metrics captured fragmentation dynamics more sensitively than spectral features. In predictive modeling, criticality measures significantly outperformed spectral power in forecasting NREM fragmentation (<i>F</i>1-score = 0.69 vs. 0.62), with the strongest gains in mild and severe insomnia subgroups.<i>Significance.</i>Insomnia is characterized by a persistent deviation from near-critical neural dynamics, reflecting compromised stability and recovery during sleep. Criticality-based EEG features provide a more mechanistic and predictive framework for identifying sleep fragmentation and may offer novel biomarkers for quantifying disrupted sleep physiology in clinical insomnia.