A mask-conditioned image generation method and its applications to underwater structural damage detection

Underwater components of hydraulic infrastructure (e.g., bridges and dams) are susceptible to damage during the service life. Compared to manual diving inspections, vision-based unmanned vehicles offer the advantage of safer, simpler, and more cost-effective inspections, attracting widespread attention in hydraulic structure monitoring. The deep learning-based vision methods depend on large-scale, high-quality datasets to achieve accurate performance. The scarcity of high-quality images and labels limits the diversity of training data, which constrains the effectiveness of vision-based detection for underwater structures. This paper proposes a two-stage mask-conditioned image generation method for constructing realistic image datasets of underwater structural damage. In the first stage, a generative adversarial network is employed to automatically generate damage masks that capture the exact shape and contour of the damage as annotations without manual labeling. In the second stage, ControlNet is utilized to improve the denoising process in a conditional generative model and generate underwater structural damage images based on the mask conditions. To validate the proposed mask-conditioned image generation method, the model is trained to generate underwater damage images, and the generated images are compared with real underwater damage images. The proposed method generates realistic damage morphologies, including undamaged surfaces, minor damage, and combined damage, with precise alignment to the input masks. The detection model trained on the generated dataset performs comparably to the model trained on an equivalent real dataset, and the accuracy of the model further improves when the dataset is expanded with the generated images. The detection results for real underwater images confirm that the model trained on generated images effectively localizes structural damage in diverse underwater environments.