Spatial–Temporal Feature Fusion with Multi-Head Attention for Supervised Anomaly Localization

Localizing anomalous activities in surveillance video requires spatial appearance to be interpreted within its temporal context. This study presents a supervised dual-branch architecture that combines YOLOv8n features from a key frame with R3D-18 features from a 16-frame RGB clip. Channel projection and spatial alignment enable convolutional fusion of the two representations. Residual multi-head self-attention then models relationships among spatial positions in the fused feature map. Dense prediction heads estimate activity classes, objectness scores, and bounding boxes on the key frame. On the six categories of UCF-Crime2Local, the reference run without nonmaximum suppression (NMS) achieves 54.21% precision, 57.85% recall, and 53.44% mAP50. Across three training seeds, mean no-NMS mAP50 is 53.44±0.46%. The finalized pipeline with class-aware NMS at IoU 0.5 achieves 54.10±0.50% mAP50 across the same seeds. Values following ± denote sample standard deviations. The complete model exceeds the fusion-without-attention variant by 4.44 mAP percentage points under the common experimental protocol. Class-wise analysis, controlled component comparisons, repeated runs, and video-level uncertainty analysis characterize the model within the specified key-frame localization task. The accompanying implementation documents temporal sampling, annotation validation, optimization, checkpoint selection, and prediction export. Continuous-video evaluation derives event intervals from category-specific frame scores, achieving 57.78% event F1 at temporal IoU 0.3 and a median onset delay of 0.80 s among matched events. These intervals are obtained through temporal postprocessing rather than a learned boundary-prediction head. Cross-dataset generalization and live-stream deployment remain to be evaluated.

Authors

Institutions

Publication Details

Journal
Sensors
Published
2026-11-12
DOI
https://doi.org/10.3390/s26227153
Primary Topic
Anomaly Detection Techniques and Applications
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Spatial–Temporal Feature Fusion with Multi-Head Attention for Supervised Anomaly Localization

Farrukh Arslan, Muhammad Naeem, Arfan Jaffar, Saba Ramzan et al.
Sensors
Anomaly Detection Techniques and Applications
article

Spatial–Temporal Feature Fusion with Multi-Head Attention for Supervised Anomaly Localization

Farrukh Arslan, Muhammad Naeem, Arfan Jaffar, Saba Ramzan, Sajid Ullah Khan, Muhammad Javaid Iqbal
article en

Abstract

Localizing anomalous activities in surveillance video requires spatial appearance to be interpreted within its temporal context. This study presents a supervised dual-branch architecture that combines YOLOv8n features from a key frame with R3D-18 features from a 16-frame RGB clip. Channel projection and spatial alignment enable convolutional fusion of the two representations. Residual multi-head self-attention then models relationships among spatial positions in the fused feature map. Dense prediction heads estimate activity classes, objectness scores, and bounding boxes on the key frame. On the six categories of UCF-Crime2Local, the reference run without nonmaximum suppression (NMS) achieves 54.21% precision, 57.85% recall, and 53.44% mAP50. Across three training seeds, mean no-NMS mAP50 is 53.44±0.46%. The finalized pipeline with class-aware NMS at IoU 0.5 achieves 54.10±0.50% mAP50 across the same seeds. Values following ± denote sample standard deviations. The complete model exceeds the fusion-without-attention variant by 4.44 mAP percentage points under the common experimental protocol. Class-wise analysis, controlled component comparisons, repeated runs, and video-level uncertainty analysis characterize the model within the specified key-frame localization task. The accompanying implementation documents temporal sampling, annotation validation, optimization, checkpoint selection, and prediction export. Continuous-video evaluation derives event intervals from category-specific frame scores, achieving 57.78% event F1 at temporal IoU 0.3 and a median onset delay of 0.80 s among matched events. These intervals are obtained through temporal postprocessing rather than a learned boundary-prediction head. Cross-dataset generalization and live-stream deployment remain to be evaluated.

SensorsVol. 26(22)
University of Engineering and Technology Lahore (PK), Superior University (PK), Lahore Garrison University (PK), Mälardalen University (SE), Mälardalen University (SE)
Anomaly Detection Techniques and Applications
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon