Simulating Marine Snow Images: Pipeline, Data Set, and Benchmark

Marine snow causes light spots and veil-like blurs in underwater images due to scattering and degrades their quality. Therefore, removing this type of noise from underwater images is necessary. However, the lack of valid and diverse marine snow image data sets has impeded the development of machine learning-based algorithms. To tackle this issue, we propose a pipeline named <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">OmniMSI</i> for generating marine snow images. This pipeline takes into account the diverse morphologies of marine snow particles and their natural distribution patterns. Specifically, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">OmniMSI</i> initiates from collecting large-scale marine snow particle images in natural environments, classifying them by morphology, analyzing their distribution patterns, and thus creating diverse marine snow masks. The pipeline then overlays the masks on real-world underwater images without marine snow and employs the designed post-process to synthesize marine snow images. Leveraging this pipeline, we create a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">M</i>ulti-morphology and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">M</i>ultidistribution <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">M</i>arine <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">S</i>now image <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">D</i>ataset (<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MM-MSD</i>), including 6000 pairs of marine snow images and corresponding reference images. We also present a novel marine snow removal method—<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">m</i>arine <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">s</i>now-<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a</i>ware <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">diff</i>usion, which can accurately estimate the location of marine snow in images and remove it progressively. Based on <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MM-MSD</i>, we conduct a comprehensive study of the state-of-the-art algorithms qualitatively and quantitatively, developing a new benchmark for marine snow removal. The benchmark evaluations demonstrate the performance and shortcomings of existing algorithms, paving the way for further research in marine snow removal. The data set and code will be released.