A Human-Centric Digital Twin: Deep Learning-Based Recognition of Human Operation Scenarios for Robotics-Assisted Power Grid Monitoring

An accurate recognition of human-led operation scenarios in power grids is pivotal for developing responsive human-robot collaborative systems, enhancing operational safety, and enabling intelligent robotic assistance. This study proposes a novel deep learning framework for interpreting such scenarios from multisensor data. An optimized Bidirectional Long Short-Term Memory network is employed to decode the bidirectional temporal dependencies inherent in human operation sequences, with an attention mechanism pinpointing critical action phases. To model the structural context of the operational environment, a pruned Graph Neural Network efficiently captures the topological relationships within the grid, providing essential spatial features for robotic path or task planning. Fusing these temporal and spatial models enables a digital twin capable of robust, multi-source data integration for understanding human actions. Experimental results demonstrate that the proposed algorithm achieves a peak accuracy of 98.2% and an F1 score of 0.96, with an AUC of 0.92, significantly outperforming baseline methods. In classifying and predicting human-executed grid operation scenarios, the model average accuracy rate reaches 98.6% — an improvement of 8.3% to 12.5% over existing models — while maintaining a sub-53[Formula: see text]s response time suitable for near-real-time robotic feedback. These findings validate the framework’s strong performance and adaptability, offering a foundation for advanced humanoid robotic applications in state monitoring, fault warning, and intelligent scheduling of power systems.