Machine learning based learning user behavior classification and recognition algorithm under multimodal data

In multimodal learning, user behavior data has heterogeneity and semantic gaps in semantic representation, which makes it difficult to accurately extract behavior features of different categories of learning users and reduces the reliability of behavior classification and recognition. Therefore, a machine learning based algorithm for learning user behavior classification and recognition is proposed under multimodal data. Utilizing video motion capture systems and audio sensors to accurately capture and fuse multimodal data, including actions and speech, to accurately understand and learn user behavior and emotional states. Adopting wavelet analysis method for denoising preprocessing of multimodal learning user behavior data to improve data amplitude stability. Using a combination of deep belief networks and support vector machines, feature extraction and classification recognition are performed on the denoised multimodal data of user behavior learning. By adopting unsupervised pre training and fine-tuning strategies, deep belief network models can better learn the inherent patterns of data during the training process, thereby accurately extracting the behavioral features of learning users and improving the reliability of behavior classification. The experimental results show that the user behavior classification algorithm based on machine learning performs well: the information gain is as high as 0.95, the feature extraction is accurate and stable, and the cross modal behavior consistency is stable at around 0.9.