A Neuro-Symbolic Spatiotemporal Vision Transformer for Zero-Shot Safety Behavior Recognition
This study evaluates video-level behavior recognition on the seven-class Construction Meta Action (CMA) benchmark using a neuro-symbolic spatiotemporal Vision Transformer. An MViTv2-S backbone feeds a neural classification head and eight weakly supervised predicate variables, which are scored against fixed class prototypes; a temporal extension uses six log-domain operators over four ordered segments. On 540 official test videos, the fused model achieves 93.35% Macro-AP, compared with 93.23% for the neural branch. Zero-shot recognition is evaluated as leave-one-class-out transfer within CMA: the target class is excluded from task-specific head training and model selection, while Kinetics pretraining and target-class prototypes remain available. Mean held-out AP is 47.66% for the rule branch and 44.69% for the neural branch, without a statistically established difference. Log-domain computation maintains finite outputs and gradients under the reported stress conditions. The measured offline cost is 199.36 ms/video for single-clip file processing and approximately 1.17 s/video for four-segment extraction on an RTX 3080 Ti laptop GPU. The findings support inspection of prototype-based score construction and numerical behavior within the evaluated benchmark. Predicate semantics are not independently annotated, predictions are not attributed to individual workers, and cross-site generalization and live-stream performance have not been validated.
Authors
- Zhengwei Chang (ORCID: https://orcid.org/0009-0008-8872-7090)
- Rui Song
- Yi Zhang
- Fangzhou Hu
- Linghao Zhang
- Ankai Zhang
Institutions
- China Electric Power Research Institute (CN)
- State Grid Sichuan Electric Power Corporation (China) (CN)
Publication Details
- Journal
- Applied Sciences
- Published
- 2026-11-12
- DOI
- https://doi.org/10.3390/app162211227
- Primary Topic
- Human Pose and Action Recognition
- Type
- article
- Field-Weighted Citation Impact
- 0.00
Funders
- State Grid Corporation of China