Substation Inspection Safety Risk Identification Based on Synthetic Data and Spatiotemporal Action Detection

During substation inspection, operators are often exposed to hazardous working environments. It is necessary to use visual sensors to determine work status and perform action detection to distinguish between normal and dangerous actions in order to ensure the safety of operators. However, due to information security, privacy protection, and the rarity of dangerous scenarios, there is a scarcity of related visual action datasets. To address this issue, this study first introduces a virtual work platform, which includes a controller for the parameterized control of scenarios and human resources. It can simulate realistic substation inspection operations and generate synthetic action datasets using domain randomization and behavior tree logic. Subsequently, a spatiotemporal action detection algorithm is utilized for action detection, employing YOLOv8 as the human detector, Vision Transformer as the backbone network, and SlowFast as the action detection architecture. Model training is conducted using three datasets: a real dataset, a synthetic dataset generated via a VWP, and a mixed dataset comprising both real and synthetic data. Finally, using the model trained on the real dataset as a baseline, the evaluation results on the test set shows that the use of synthetic datasets in training improves the model's average precision by up to 10.7%, with a maximum average precision of 73.61%. This demonstrates the feasibility, effectiveness, and robustness of synthetic data.