Masticatory organ dysfunction recognition based on multimodal information

This paper describes research on designing a system for initial masticatory dysfunction recognition, a study with significant implications for patient treatment. The input contains three sources of information: text from questionnaires, videos, and face images acquired using mobile applications. We collected a dataset of 86 patients. Because of the relatively small dataset, we used binary classification in this research. We applied three classic machine learning models: Regularized Logistic Regression (RLR), Support Vector Machine (SVM), and Extreme Gradient Boosting (XGB). We designed several features acquired from video and images and selected some using different selection methods. In the experimental study, we assessed which source of information is the most discriminative for this classification. We achieved the highest F1 measure (macro) using textual data. Classification results based on dynamic features from video achieved 0.643 F1 measure (macro). Features from static images represent the worst predictive value, and the best result is 0.5694. This result does not support our hypothesis that facial asymmetry is essential in diagnosing masticatory disorders. However, the identified asymmetry is the basis for further tests to correct posture problems and additional tests by a physiotherapist, which underscores the practical implications of our research on the patient’s treatment. • Text from questionnaires is key for identifying masticatory dysfunction. • Dynamic features based on jaw movement support text in masticatory dysfunction recognition. • Face asymmetry has a low value for masticatory recognition but aids other issues. • Only the null hypothesis of neuroticism-masticatory correlation from the Big Five could not be rejected.