Comparison of Different Dissimilarity Measures in Representational Similarity Analysis of Multimodal EEG-fMRI Data

The use of representational similarity analysis (RSA) methods to investigate information representation in the brain has become increasingly popular. One of RSA's major advantages is its ability to match and compare information across different brain assessment modalities. Typically, dissimilarity between brain activity patterns is measured using methods based on correlation or the accuracy of a linear classifier. Although various distance measures are available, selecting the optimal distance measure for each modality requires assessing each measure's reliability and repeatability. To address this, we used an EEG-fMRI dataset collected during an image classification task to determine the most suitable distance measures for each modality. For the EEG data, we used five dissimilarity measures to build representational dissimilarity matrices (RDMs). For fMRI data, we constructed RDMs using four dissimilarity measures. To identify the best measure for each modality, the runs for each subject have been divided into even and odd groups. Then we utilized Pearson and Spearman correlation to evaluate reliability of each measure between odd and even runs. Our results showed that, for the fMRI modality, Pearson correlation is the most reliable measure. For the EEG data, the most reliable distance measure is SVM accuracy. Additionally, we used multivariate noise normalization (MNN) to improve performance across all criteria in both modalities. Overall, we recommend to use MNN as a preprocessing step for both modalities. Also, in simultaneously recorded EEG-fMRI studies we advise to use the SVM accuracy for constructing RDMs for EEG, and Pearson correlation for building fMRI RDMs.