A Dual-Branch ST-GCN System for Joint Recognition of OOW Unsafe Behaviors and Facial Fatigue Features

The Officer on Watch (OOW) is critical to ensuring the safety of the vessel, cargo, and crew during navigation. To reduce maritime accidents caused by unsafe behaviors or fatigue, this paper proposes a dual-branch detection system based on Spatial–Temporal Graph Convolutional Networks (ST-GCN): BODY-ST-GCN for pose-based behavior recognition and FACE-ST-GCN for facial state analysis. For spatial modeling, a Triple Graph Fusion (TGF) strategy is introduced to integrate static, adaptive, and attention graphs, enhancing the representation of skeletal and facial keypoints. For temporal modeling, BODY-ST-GCN incorporates a Three-Scale Parallel Temporal Convolutional Network (TSP-TCN) to capture multi-scale motion dynamics, while FACE-ST-GCN uses a Temporal Adaptive Module (TAM) to extract stable facial state features. Furthermore, a joint risk classification mechanism categorizes OOW duty states into four hierarchical levels: Safe, Early Fatigue Warning, High Fatigue Risk, and Emergency. This mechanism enables continuous, real-time monitoring and dynamic assessment. Experiments demonstrate that BODY-ST-GCN and FACE-ST-GCN achieve macro average precisions of 0.969 and 0.947, respectively, outperforming the baseline ST-GCN by 6.4% and 14.9%, providing reliable technical support for onboard safety management.