An Integrated Real-Time Approach to Mitigate Noncompliant Behavior in Mobile Crane Lifting Operations

The command-operation interaction is crucial for ensuring safety and efficiency in mobile crane lifting operations. Noncompliance by crane operators, particularly failing to follow hand signals, represents a significant failure in this interaction and is known to cause accidents. This research addresses the following question: How can we accurately detect noncompliant behavior (i.e., failing to follow hand signals) in real-time during mobile crane lifting operations? In addressing this research question, we propose an integrated real-time approach to detect people failing to follow hand signals to mitigate noncompliant behavior. The proposed approach detects noncompliant behaviors by assessing the consistency between hand signals recognized by computer vision and the crane motion poses estimated by nonvision sensors. The feasibility and effectiveness of our approach were demonstrated using a real-life case study in Wuhan, China. The results indicated that our proposed approach achieved an accuracy of 95.09% and a latency of 475 ms in the detection of operators failing to follow hand signals. Our approach can help mobile crane operators correct noncompliant behaviors and improve operational safety. We suggest that our approach can be deployed in environments with limited computational resources.