Machine Learning Methods for Multimodal Classification of Personality Traits Using Social Media Data

In this paper, we investigate the effectiveness of various machine learning methods for the multimodal classification of personality traits using the HEXACO model and open data from social media users. Particular attention is paid to the analysis of data obtained from user profiles, including statistical, visual, and audio characteristics. For each personality trait from the HEXACO model, a machine learning model is trained that determines the manifestation degree of this trait (low, medium, high). The influence of different types of information, both individually and in combination, on the weighted F1-score is considered. The study provides a comparative analysis of traditional machine learning methods that use all features and approaches with the selection of the most significant features. A strategy for combining data from different modalities using early merger is evaluated. The experimental results demonstrate that early merger models using selected features from different modalities provide the best result of the weighted F1-score, which varies depending on the personality trait from 0.76 to 0.85.