Impact of Real-time Map Feedback on Coordinated Image Sampling for 3D Reconstruction

Efficient image sampling from diverse viewing angles is crucial for high-quality 3D map reconstruction, and coverage control provides a promising solution to this mission. Moreover, with the recent advancements in real-time 3D reconstruction algorithms, it is now possible to iteratively reconstruct 3D maps in real time, providing immediate map feedback to robot motion control. In this paper, we propose a novel coordinated image sampling algorithm that leverages real-time map feedback to enhance the quality of the reconstructed 3D model. We first formulate the problem as an angle-aware coverage control problem, where images are captured from multiple angles across the field of interest by drones. These images are processed in real time by so-called NeuralRecon to generate an evolving 3D mesh of the environment. Mesh changes across the field serve as feedback to update the importance index of the coverage control as the map evolves. We then design a QP-based controller to certify a sampling performance by constraining the decay rate of the objective function. Simulations in Unity and ROS2 demonstrate that our feedback-driven approach outperforms the conventional method without map feedback, resulting in a more complete and accurate 3D map. Project page: https://htnk-lab.github.io/map_feedback_coverage/.