Point Cloud Instance Segmentation for Reducing Effort in Oil and Gas Digital Twin Construction

Abstract 3D point cloud data captured from Oil and Gas (O&G) facilities are becoming increasingly abundant. This data supports the interactive visualization of complex plants, which is central to numerous industrial advancements. As industries seek to innovate and optimize their processes, the versatile use of 3D point cloud data is proving invaluable beyond traditional O&G applications. The precision and comprehensiveness of 3D point cloud data facilitate efficient design, construction, maintenance and operational monitoring across various industrial sectors. Digital Twin (DT) solutions enable this by maintaining 3D representations, such as 3D CAD models, that accurately reflect the current state of facilities. As installations change, the process of 3D scanning is often repeated to capture the latest updates. Engineers and designers then use modelling software to visualize the resulting point cloud and update the facility's 3D representation, incorporating any new modifications. However, this involves interpreting the visual point cloud data and structuring it into meaningful objects (such as valves, tanks, etc.), a process that remains manual, requiring labor-intensive tasks that are prone to errors and incur significant costs due to the need for specialized labor. We propose an Artificial Intelligence approach, that uses a deep learning-based algorithm to automatically segment 3D objects within a point cloud. Our workflow recognizes instances of usual components of production systems, such as control valves, pipes, and transmitters. A state-of-the-art 3D instance segmentation approach is adapted to the O&G domain. Given the lack of publicly available instance segmentation datasets in this domain, we manually annotate 54 scenes from an O&G offshore plant, each of them around 20 m2 in footprint. To further enhance model training, we also incorporated synthetic point clouds. Practical results on real point clouds, acquired from operational facilities and pre-processed as required by commercial DT solutions, indicated a performance of approximately 70% success (f1-score) in segmenting components of O&G production systems, reaching up to 85% for components such as control valves. Accordingly, a 52.4% reduction in costs and efforts associated with manually interpreting 3D point clouds is estimated. Point cloud instance segmentation has attracted much attention in the computer vision research community, and many approaches have been proposed. However, results in the literature are heavily biased toward a few benchmarking datasets, which were acquired in indoor environments like offices and rooms. Few studies have addressed the O&G domain, creating a gap in the literature for achieving more competitive results. This lack of comprehensive research limits the potential for innovation and optimization in the industry. This paper contributes by (i) assessing the performance of state-of-the-art algorithms in the domain of O&G facilities, (ii) proposing modifications to better generalize to components of O&G production systems, (iii) objectively measuring the impact of this algorithm in reducing manual effort for engineers, and (iv) advancing towards a new DT functionality that derives an asset 3D representation, that is faithful to physical reality, through the automatic interpretation of point cloud scans.