MEGWeaver: Sentence Relational Pretraining for Audio-Free MEG-to-Text Decoding
Decoding perceived language from non-invasive magnetoencephalography (MEG) offers a promising route toward brain-to-text communication. NeuSpeech first introduces Whisper to MEG-to-text decoding, while BrainECHO improves its accuracy through acoustic reconstruction. However, whether further accuracy gains can be achieved without relying on additional acoustic supervision remains unclear. Here, we propose MEGWeaver, a framework for direct MEG-to-text generation that pretrains a MEG encoder based on sentence relational pretraining (SRP), which aligns the in-batch similarity structure of MEG representations with sentence-level relations defined by similarities between embeddings from a frozen sentence encoder. Only the pretrained sensor projection is transferred to downstream text decoding. Across English, Dutch and Chinese datasets under held-out-participant evaluation, MEGWeaver achieves BLEU-4 scores of $75.10\pm1.21$, $48.78\pm1.17$ and $39.07\pm0.72$, respectively, outperforming NeuSpeech and sentence retrieval across all three datasets and obtaining numerically higher BLEU-4 than our BrainECHO reproduction on Dutch and Chinese. These findings support direct MEG-to-text generation for known sentences under the present segmentation protocol, while decoding novel sentences remains an open problem.
Authors
- Xiaoling Wang (ORCID: https://orcid.org/0000-0002-5396-2284)
- Huanran Zheng (ORCID: https://orcid.org/0009-0005-6261-6702)
- Dell Zhang
- Chenyuan Zhu
Institutions
- East China Normal University (CN)
- Northeastern University (CN)
Publication Details
- Published
- 2026-11-17
- DOI
- https://doi.org/10.66977/xsci.2611.002u
- Primary Topic
- Speech Recognition and Synthesis
- Type
- preprint