Dual-modal Data Attention Recognition Based on Multi-Head Self-Attention Mechanism

Attention is one of the important cognitive functions of human beings, but with the prevalence of students' lack of attention, especially in the early childhood, the cultivation of attention becomes particularly critical thing. Electroencephalogram (EEG) signals can reflect the cognitive state of the brain, especially different frequency bands such as α, β, θ, δ bands are closely related to attention. Eye tracking data and attention are directly and closely related. Eye tracking data and EEG are complementary, which can reveal the characteristics of human visual attention distribution, attention transfer and distribution. Relevant studies show that the combination of EEG and eye tracking technology can more accurately assess and adjust attention. In this paper, a two-modal feature fusion model based on multi-head self-attention mechanism is proposed to learn the relationship between the two-modal features data. We input the features of multi-head self-attention fusion, the features of single mode and the features of bimodal directly splintered into multiple classifiers, and the features of multi-head self-attention fusion obtain higher classification accuracy.