An Enhanced Multimodal Video Violence Detection System via Feature Fusion of Visual, Audio, and Textual Information

With the recent increase in video content consumption, the importance of technology that automatically detects violent scenes has grown. In this paper, we propose a multimodal approach that integrates visual, audio, and textual information to overcome the limitations of previous studies that primarily relied on visual data. We extend the TEVAD model, a dual-modal model utilizing visual and textual information, by incorporating multimodal data that includes visual, audio, and textual inputs. We conducted experiments on various feature fusion methods using the XD-Violence dataset, which includes audio information for violence detection. The experimental results demonstrate that the proposed multimodal approach achieves an average precision (AP) of 82.2%, outperforming single-modal (visual: 75.6%, audio: 64.3%, textual: 28.4%) and dual-modal approaches (visual+audio: 80.7%, visual+textual: 78.0%). These results indicate that the multimodal approach is more effective in enhancing violence detection performance compared to traditional single and dual-modal approaches.