Deep learning-based on-site identification and volume measurement of bulk material in construction industry

Bulk materials are important raw construction materials, the adequate and precise supply of which enables a smooth construction process. Two conventional techniques for controlling the quantity of bulk materials on-site along the supply chain are: 1) estimation based on the cone-shape of the material pile, but the accuracy is low; 2) calculation by using bulk density and weighing stations, which are not available in all facilities. To address this issue, we propose a novel hybrid camera-based method combining a red-green-blue (RGB) camera and a light detection and ranging (LiDAR) sensor for automatic material type identification and volume measurement. The data from two-dimensional pictures and three-dimensional point cloud were segmented and extracted with two Deep Learning models: “You Only Look Once“ (YOLO) v5 and PointNet++for the identification of material type and volume measurement. This novel hybrid camera-based method was developed as an Industry 4.0 solution to enable the automatic and accurate evaluation of the volume of bulk materials. With a precision of 81.6 % in object recognition and a volume estimation deviation of less than 8 %, it provides a reliable and efficient alternative to conventional, labour-intensive measurement techniques. • DL-based on-site identification and volume measurement of construction materials. • Combining 2D and 3D data processing for ensuring high efficiency and flexibility. • Low deviation between the true volume and its value calculated by the given method. • The high accuracy achieved is independent of the bulk material's piling condition.