Real-Time On-Site Indoor Modeling and Quality Visualization Using Mixed Reality Device with TOF Sensor

In this paper, we propose a real-time on-site indoor modeling and modeling quality visualization method using a Mixed Reality (MR) device with a Time-of-Flight (ToF) sensor to realize efficient reliable indoor model generation. The modeling method is based on the rule-based fast voxel labeling and consists of two steps, i.e., local modeling and global modeling. In the local modeling, space occupancy labels and attribute labels of ceiling, wall, wall opening, wall candidate, floor, object, and space are assigned to each cell of the local voxel from point clouds in each frame of laser scanning by the MR device. In the global modeling, the results of the local modeling are integrated with the global voxel considering the label assignment quality based on entropy and probability. The quality of the resulting model is shown to the user using MR visualization to check the modeling results. For real-time visualization of the quality of modeling results, a simple textured polygon model is created and used in the MR visualization. The polygon model is efficiently generated from the local and global voxels, and the texture represents the quality of the modeling results. Our method was applied to a real indoor environment, and its performance was evaluated. The modeling framerate was 2fps, and the precision for labeling was 98%. The polygon model for MR visualization was created and updated within 20msec, and real-time MR visualization could be achieved. These results showed that the real-time indoor modeling and quality check on-site using the MR device with a ToF sensor could be realized.