STFA-Net: Spatio-Temporal Feature Fusion with Adaptive Attention Modulation for Robust Dynamic Object Recognition

Abstract Recognizing objects that move rapidly and change posture drastically remains a bottleneck for robotic perception, autonomous driving, and campus logistics platforms operating among pedestrians, vehicles, and other dynamic agents. This setting couples an appearance problem with a temporal one: high-speed motion and out-of-plane rotation destroy the pixel-level stationarity that static detectors rely on, while illumination drift, partial occlusion, and cluttered backgrounds inject non-stationary distractors. This paper proposes STFA-Net, a unified recognition framework built around a spatio-temporal feature fusion and attention-adaptive modulation mechanism. First, a multi-scale dilated convolutional encoder is coupled with a lightweight temporal module that combines shifted temporal aggregation with gated recurrent refinement, so that target representation and motion patterns are learned jointly rather than in separate streams; the temporal module accounts for less than 8% of total FLOPs. Second, a Channel–Spatio-Temporal Dual-path Attention (CSTA-DA) module allocates weights dynamically according to measured feature contribution: a channel path models what is informative, a spatio-temporal path models where and when it is informative, and a contribution-aware gating vector arbitrates between the two paths for every input sample, enabling the model to autonomously focus on informative regions and suppress illumination change, occlusion, and background interference. Third, a feature recalibration mechanism re-scales the fused representation with a contribution-driven gain and an explicit redundancy-sparsity penalty, strengthening discriminative channels while suppressing redundant ones. Training data are produced by a physics-accurate synthetic data engine built on NVIDIA Isaac Sim with randomized cameras, lighting, and physics-driven object trajectories, and a fragility-driven randomization protocol that targets the corruption channels our own sensitivity study shows actually break recognition — exposure shift above all (Section 5.4). On ImageNet-Vid, STFA-Net reaches 64.5% [email protected], exceeding the strongest compared baseline by 0.8 points with 31% fewer parameters and 58% fewer FLOPs. On our ST-Warehouse benchmark of 14,400 frames with pixel-accurate labels, STFA-Net reaches 81.8% [email protected] (3.4 points above the strongest baseline, TransVOD), a counting mean absolute error of 0.35, and retains 88.3% of its clean accuracy under the combined sensor-realistic corruption, versus 76.1% for the strongest baseline (TransVOD). Component ablations attribute + 6.9 points to the proposed modules on identical training schedules, and logged gate statistics confirm the predicted input-adaptive behaviour: the gate value rises under photometric corruption (mean α = 0.68) and falls under geometric corruption (mean α = 0.34). All benchmark statistics, corruption protocols, and engine-cost measurements are reported with the failure modes that silent synthetic pipelines usually hide.