Diagnosing Motor-Imagery BCI Failure: Separating Methodological from Neural Decoding Failure

Objective. Motor-imagery brain–computer interfaces (BCIs) fail to provide reliable control for many users, a phenomenon called BCI illiteracy. Instead of trying to rescue all poor performers with stronger decoding, this study asks whether failure can be diagnosed beforehand, separating cases where the signal contains usable information but the decoder misses it (methodological failure) from cases where the signal itself lacks sufficient discriminable structure (neural failure).Approach. Using BCI Competition IV Dataset 2a (nine subjects, four-class motor imagery, left-mastoid reference, no artifact rejection), a two-step diagnostic framework was applied. Step 1 computed a classifier-free Riemannian separability index from session-T covariance matrices. Step 2 used paired permutation tests to evaluate individual gains of filter-bank CSP (FBCSP) over standard CSP+LDA, with a Riemannian minimum-distance classifier (MDM) as a geometry-matched control.Main results. The separability index was strongly associated with decoding performance (r = .89) and served as an in-sample proxy for signal quality, and was robust across covariance estimators and referencing schemes. However, the index alone could not distinguish the three worst subjects (S2, S5, S6), who all clustered at low separability. FBCSP recovered S5 (κ from .28 to .55, p < .001), while S2 and S6 remained low across pipelines. S6's gain became significant only after ocular-artifact removal (κ gain +.185, p = .002), highlighting the sensitivity of the second-step diagnosis to preprocessing.Significance. BCI illiteracy conflates at least two failure modes. The proposed two-step approach, a geometry-based screen followed by a statistically rigorous recoverability test, offers a cautious, individually-grounded diagnostic that respects the limits of the tools.