Conversion of Point Cloud Data to 3D Models Using PointNet++ and Transformer

This work presents an approach to reconstructing 3D models from point cloud data, based on the use of modern neural network architectures. The basis of the method is PointNet++ and Transformer. PointNet++ plays a central role, providing efficient feature extraction and encoding of complex 3D scene geometries. This is achieved by recursively applying PointNet++ to nested partitions of the input point set in metric space. Convex decomposition, an important step in the approach, allows transforming complex three-dimensional objects into a set of simpler convex shapes. This simplifies data processing and makes the reconstruction process more manageable. The Transformer then trains the model on these features, allowing for the generation of high-quality reconstructions. It is important to note that the Transformer is used exclusively to determine the position of walls and object boundaries. This combination of technologies allows achieving high accuracy in the reconstruction of 3D models. The main idea of the method is to segment the point cloud into small fragments, which are then restored as polygonal meshes. To restore missing points in point cloud data, a method based on the L1-Median algorithm and local point cloud features is used. This approach can adapt to various geometric structures and correct topological connection errors. The proposed method was compared with several modern approaches and showed its potential in various fields, including architecture, engineering, digitization of cultural heritage, and augmented and mixed reality systems. This underscores its' wide applicability and significant potential for further development and application in various fields.