Integrating Object Detection in Bird-View Systems for Enhancing Safety in Construction Machinery

Working with construction machinery necessitates that workers remain attentive to effectively survey their surroundings. The complex environment, blind spots, and limited visibility while simultaneously tackling complex tasks create hazardous conditions. Camera systems providing a bird view help monitoring the machine’s surroundings, but still require active monitoring while working. In this paper, we propose extending bird-view systems with object detection. We investigate and evaluate two different approaches: Detecting people in four fisheye camera images and transforming the obtained bounding boxes into a bird view, and directly detecting people in bird view. Our models, based on the YOLOv5 and YOLOv8 architectures, were trained and validated on custom datasets with 3,302 fisheye images and 1,343 bird-view images. The models achieved [email protected] scores of 0.514 (fisheye) and 0.546 (bird view) and run at 25 frames per second on an NVIDIA Jetson Nano edge GPU. Our trained models for the detection utilize the YOLOv5 and YOLOv8 architecture and run inference with 25 frames per second on an NVIDIA Jetson Nano edge GPU. The first approach also addresses objects disappearing in stitching areas - characteristic of bird-view projection errors - by displaying bounding boxes even if they are invisible in bird view.