Machine and deep learning for personality traits detection: a comprehensive survey and open research challenges

Natural language processing (NLP), a prominent research domain of Artificial Intelligence (AI), analyzes users’ generated content on social media for various purposes like sentiment analysis, text summarization, chatbots, fake news detection, etc. Recent advancements in NLP have helped for analysis of human behavior analysis and predicting various human personality traits. Understanding personality traits has long been a fundamental pursuit in psychology and cognitive sciences due to its vast applications for understanding from individuals to social dynamics. Due to online social platforms where people express their views, experiences and comments, NLP is applied for users’ behavior and personality analysis, which is helpful in defining marketing strategies, consumers’ behavior analysis, team building, etc. This research study provides a comprehensive overview of existing methodologies, applications, and challenges in the field of personality traits detection using shallow machine learning, ensemble learning and deep learning. To conduct this study, recent research publications relevant to NLP for this new but emerging research domain are reviewed. The background knowledge of personality models of various nature is discussed for better domain understanding. The study encompasses machine learning and deep learning models with thorough analysis of traditional and innovative techniques including ensemble learning and transformer-based models in chronological order highlighting the trend analysis showing evolution of application of advanced methods. The review also presents and compares the widely used datasets which may guide the researchers for selection of datasets in future studies. Performance evaluation metrics have been discussed which are used in the relevant literature. Furthermore, it explores the application of research of personality traits detection in various domains highlighting its significance. We have also carried out extensive empirical analysis using conventional textual to advanced deep embedding features and applying machine learning, ensemble learning and deep learning algorithms. Finally, before conclusion, the review highlights the open research issues and challenges as potential future directions for the researchers.