Combining Vision Transformers and crane load information for a rope winding detection system

Abstract Safety is a very important issue on construction sites. In the case of cranes, the condition of the wire rope is a predominant factor that can have major consequences if problems occur. Not only is the condition of the wire rope of concern, but even the winding problem on a drum can be a problem if the wire rope is not wound correctly. The latter is the problem addressed in this paper, where the winding status of a rope is detected and analyzed by a Vision Transformer that merges load data with the image of the rope on the drum as it is wound. As soon as a potentially dangerous condition is detected, a blocking action is sent to the crane to prevent critical situations. Active supervision by a crane operator is not required, as the system stops automatically. A camera collects images of the drum at a constant interval, and a transformer model infers the individual images, and detects possible hazards. The developed condition detection system is then tested on a real crane at a construction site as a proof-of-concept. The crane has built-in sensors that can provide important data such as tower rotation, boom inclination, and boom extension. For this study, the weight of the crane hook is measured by the crane sensors and then combined with the camera image to determine the safety condition of the wire rope, helping to prevent accidents in advance by stopping the crane.