Spatial puzzle-based with multi-level visual features for self-supervised video anomaly detection

Existing video anomaly detection approaches face limitations in generalization and dependency on labeled data. We introduce spatial puzzle-based multi-level visual features (SPMVF), a self-supervised framework that integrates pose estimation, foreground segmentation, and RGB features through a multi-semantic spatial puzzle mechanism. SPMVF jointly optimizes multi-level visual cues to model fine-grained human motions, contextual object interactions, and spatial coherence. Specifically, we abstract puzzle positions through spatial pseudo-labeling to account for spatial relationships in heterogeneous feature streams. For multi-semantic spatial puzzle decryption, we formulate it as a multi-label classification problem that predicts locations based on independent puzzle pieces instead of judging full permutations. In the inference process, a confidence-based scoring system can identify anomalies through spatio-temporal consistency differences. Our extensive experiments on public benchmark datasets, including UCSD Ped2, CUHK Avenue, and ShanghaiTech. The results show that the models trained by our proposed framework demonstrate its effectiveness.