Visual saliency decoding algorithm based on EEG signals

As a fundamental characteristic of the visual system, visual saliency has become a significant focus of research in recent years. However, existing methods rely solely on single-modality EEG signals with low signal-to-noise ratios, which significantly limits decoding accuracy and exhibits poor algorithmic interpretability. This study proposes an improved VSM generation method based on a two-dimensional Gaussian distribution to optimize VSM decoding. We designed a novel deep learning model, termed Multi-Encoder Single-Decoder with Channel Attention (MESD-CA), which integrates multimodal data, including electroencephalogram (EEG) signals and visual images, for VSM decoding. The MESD-CA model employs a two-stage training strategy to disentangle category-specific information from visual saliency features in EEG signals and incorporates visual image data to enable efficient decoding. Experimental results demonstrate that the MESD-CA model achieves localization precision and effective region F1 scores of 73.8% and 68.6%, respectively, significantly outperforming the Generative Adversarial Network (GAN). In addition, analysis of channel attention weights and the construction of brain topography reveal Fp2, FC4, and PO7 as key regions associated with the task. Distinct brain patterns were also observed across different object categories and fixation positions. This study provides new insights into the functional relationship between brain activity and the visual system, offering a foundation for applications in brain-computer interface (BCI) technologies.