Reading the weights: a language model audits and repairs EEG decoders without data

EEG decoders can score well by exploiting signals other than the brain activity they claim to decode. Existing audits need data and forward passes. We ask whether a general-purpose language model, given only a trained decoder's weights and a description of its input, can say what the decoder relies on, predict how it behaves on another dataset, and repair it. Blind to data, the model identified the mechanism of eight real decoders (four EEGNets, a CBraMod and a LaBraM that read horizontal eye movements on PhysioNet, a mu-rhythm decoder, and a cue-evoked-potential decoder on BCI Competition IV 2a) and named 24 of 25 planted cues. Its verdicts predicted out-of-dataset behaviour: decoders it called eye-driven fell below chance on 52 unseen Cho2017 participants, the one it called mu-driven reached 59.7%. Editing weights directly with no training, it removed the eye shortcut from CBraMod (46.7% to 60.4% on Cho2017, 44 of 52 participants improved; an independent second agent reached 57.1%). On a sealed bench of eight cues designed by an independent model, with predictions hashed before scoring, a trivial weight statistic localised the planted signal as well as or better than the reader (20/21 vs 14/21); the reader's added value lay in naming the signal family (6/7) and a class rule. Ten publicly released decoders were audited the same way. Preprint v1, 3 November 2026. Code: https://github.com/YihengLi-1/eeg-motor-imagery-decoding