Novel Multimodal Fusion Algorithm for Non-Intrusive Anxiety Detection

Early detection of anxiety disorders in a non-intrusive manner is crucial, as these conditions can profoundly impact an individual’s health and daily functioning. Traditional approaches relying solely on unimodal data often fall short, potentially introducing bias and inaccuracies. TI-Fusion is a novel late multimodal fusion technique that integrates text and image data for a unified reliable outcome, overcoming limitations in existing methods. The primary advantage of TI-Fusion is its non-intrusive nature, ensuring patient comfort by avoiding invasive methods while still delivering robust diagnostic capabilities. The study utilizes six advanced machine learning algorithms (Gaussian Naive Bayes, XGB Classifier, K-Neighbors, SVM, Decision-Tree, and RandomForest) for data classification, pattern recognition, and predictive accuracy. Concurrently, image data from the KDEF and CK+ datasets was processed through a Convolutional Neural Network (CNN) enhanced with a Real Gabor filter, which is particularly adept at capturing textures, edges, and complex visual patterns necessary for precise image analysis and recognition. By employing a late multimodal fusion approach, TI-Fusion integrates the outcomes of models trained on distinct data modalities, yielding a more comprehensive and accurate prediction than unimodal methods. This technique not only surpasses existing multimodal approaches but also achieves a commendable final accuracy rate of 92.38%, demonstrating its effectiveness in enhancing the early detection of anxiety disorders.