MEGWeaver: Sentence Relational Pretraining for Audio-Free MEG-to-Text Decoding

Decoding perceived language from non-invasive magnetoencephalography (MEG) offers a promising route toward brain-to-text communication. NeuSpeech first introduces Whisper to MEG-to-text decoding, while BrainECHO improves its accuracy through acoustic reconstruction. However, whether further accuracy gains can be achieved without relying on additional acoustic supervision remains unclear. Here, we propose MEGWeaver, a framework for direct MEG-to-text generation that pretrains a MEG encoder based on sentence relational pretraining (SRP), which aligns the in-batch similarity structure of MEG representations with sentence-level relations defined by similarities between embeddings from a frozen sentence encoder. Only the pretrained sensor projection is transferred to downstream text decoding. Across English, Dutch and Chinese datasets under held-out-participant evaluation, MEGWeaver achieves BLEU-4 scores of $75.10\pm1.21$, $48.78\pm1.17$ and $39.07\pm0.72$, respectively, outperforming NeuSpeech and sentence retrieval across all three datasets and obtaining numerically higher BLEU-4 than our BrainECHO reproduction on Dutch and Chinese. These findings support direct MEG-to-text generation for known sentences under the present segmentation protocol, while decoding novel sentences remains an open problem.

Authors

Institutions

Publication Details

Published
2026-11-17
DOI
https://doi.org/10.66977/xsci.2611.002u
Primary Topic
Speech Recognition and Synthesis
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

MEGWeaver: Sentence Relational Pretraining for Audio-Free MEG-to-Text Decoding

Xiaoling Wang, Huanran Zheng, Dell Zhang, Chenyuan Zhu
1 links
Speech Recognition and Synthesis
preprint

MEGWeaver: Sentence Relational Pretraining for Audio-Free MEG-to-Text Decoding

Xiaoling Wang, Huanran Zheng, Dell Zhang, Chenyuan Zhu
preprint en

Abstract

Decoding perceived language from non-invasive magnetoencephalography (MEG) offers a promising route toward brain-to-text communication. NeuSpeech first introduces Whisper to MEG-to-text decoding, while BrainECHO improves its accuracy through acoustic reconstruction. However, whether further accuracy gains can be achieved without relying on additional acoustic supervision remains unclear. Here, we propose MEGWeaver, a framework for direct MEG-to-text generation that pretrains a MEG encoder based on sentence relational pretraining (SRP), which aligns the in-batch similarity structure of MEG representations with sentence-level relations defined by similarities between embeddings from a frozen sentence encoder. Only the pretrained sensor projection is transferred to downstream text decoding. Across English, Dutch and Chinese datasets under held-out-participant evaluation, MEGWeaver achieves BLEU-4 scores of $75.10\pm1.21$, $48.78\pm1.17$ and $39.07\pm0.72$, respectively, outperforming NeuSpeech and sentence retrieval across all three datasets and obtaining numerically higher BLEU-4 than our BrainECHO reproduction on Dutch and Chinese. These findings support direct MEG-to-text generation for known sentences under the present segmentation protocol, while decoding novel sentences remains an open problem.

East China Normal University (CN), Northeastern University (CN)
Speech Recognition and Synthesis
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon