Multimodal Sentiment Analysis and Depression Detection of Social Media Users Based on MLP

Multimodal sentiment analysis and depression detection are two prominent research topics. Multimodal sentiment analysis aims to identify the underlying emotions by analyzing information from different modalities, such as text, images, and audio. Depression detection aims to leverage multimodal sentiment analysis to regress depressive tendencies over an extended temporal sequence. This study focuses on models composed entirely of multilayer perceptrons (MLPs) at the stage of multimodal feature fusion. Such models typically mitigate the high computational complexity associated with attention-based architectures while maintaining efficiency to a certain extent. This study uses CubeMLP, a model entirely composed of MLP, which can be used for multimodal sentiment analysis and depression detection. The experiments are conducted on the MOSI and MOSEI datasets. The model is validated on both datasets, achieving MAE scores of 0.811 and 0.552 on the test sets, respectively. These results demonstrate the model's simplicity and efficiency. The study also conducts a hyperparameter analysis on the architecture of CubeMLP, leading to three key conclusions that enhance the model's interpretability to a certain extent. This study further extends the structure of the fundamental MLP units and conducts experiments incorporating different activation functions. Finally, this study presents two potential application scenarios for the model. This study aims to enhance the understanding of pure MLP-based models and contribute to advancements in multimodal sentiment analysis and depression detection.