Target Face Recognition and Multimodal Classification Method in Video

Automatic identification and processing of specific people in videos has wide applications in intelligent monitoring, public opinion analysis, and multimodal sentiment computing. However, existing methods often have problems such as low efficiency and poor robustness when processing long and multiperson mixed video content, and there is no complete tool for target person facial analysis. To this end, we propose a processing framework for target people in videos, aiming to achieve accurate identification, temporal tracking, and facial segment cropping, and combine multimodal features for facial classification. The system consists of two core modules: one is the target person processing module based on the model YOLOv8n-face and ArcFace, combined with the Intersection over Union matching tracking strategy; the other is a multimodal face classification network that integrates image and structured features, which is suitable for small sample recognition tasks. Video experiments cover a variety of resolution and frame rate scenarios. The results show that the system can stably extract the target person's facial image, with an average processing time of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathbf{0. 2 - 0. 6}$</tex> seconds per frame, and the tracking module has minimal overhead. The classification task builds a multimodal training set based on AffectNet. Under the condition of limited samples, the verification accuracy of the fusion model is better than that of the structured feature model, showing strong generalization potential. This study constructs an efficient and scalable target person processing and classification framework, which provides a practical path for character behavior analysis and emotion understanding in largescale videos.