Indoor environment monitoring method based on the fusion of audio recognition and video patrol features

Abstract To accurately detect the abnormal behavior of business hall employees in the indoor working environment and ensure the normal operation of the store, a deep learning technology is combined to build an audio and video feature extraction network. First, a separate video and an audio feature extraction model are built by using a convolutional neural network and a long short-term memory network, and then an abnormal behavior detection model of audio and video feature fusion is built using multi-channel modules. This model can divide audio and video data into multiple different channels, and perform feature extraction and recognition on each channel. To assess the performance and robustness of the proposed model, experiments are organized on multiple datasets. The research outcomes denote that compared with some existing neural network models, the detection accuracy, recall, F1 value, and mean average precision values of the proposed model in the video audio dataset are 97.85, 96.98, 95.61, and 35.61%, respectively. The research method has a remarkable fusion effect on video and audio datasets, which improves the accuracy and robustness of abnormal behavior recognition. In addition, user and expert satisfaction with the model has also improved significantly, reaching 0.92 and 0.97, respectively. The novelty of this study lies in the comprehensive analysis of audio and video data by using multi-channel processing technology. Through effective extraction and fusion of video frames and audio signals at the same time, it significantly improves the detection ability of abnormal behaviors in indoor environments in complex scenes.