HKCoral: Benchmark for Dense Coral Growth Form Segmentation in the Wild

Underwater coral reef monitoring plays an important role in the maintenance and protection of the underwater ecosystem. Extracting information from the collected coral reef images and videos based on computer vision techniques has recently gained increasing attention. Semantic segmentation, which assigns semantic category information to each pixel in images, has been introduced to understand coral reefs. Satisfactory semantic segmentation performance has been achieved based on large-scale in-air data sets with densely labeled annotations. However, underwater coral reef understanding is less explored and existing underwater coral reef data sets are mainly captured under <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ideal</i> and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">normal</i> conditions and lack variance. They cannot fully reflect the diversity and properties of coral reefs. Thus, trained coral reef segmentation models show very limited performance when deployed in <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">practical</i>, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">challenging</i>, and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">adverse</i> conditions. To address these issues, in this article, we propose an <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">in-the-wild</i> coral reef data set named <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">HKCoral</i> to close the gap for performing in-situ coral reef monitoring. The collected data set with dense pixel-wise annotations possesses larger diversity, appearance, viewpoint, and visibility variations. Besides, we adopt the fundamental coral <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">growth form</i> as the foundation of our semantic coral reef segmentation, which enables a strong generalizability to unseen coral reef images from different sites. We benchmark the coral reef segmentation performance of 17 state-of-the-art semantic segmentation algorithms (including the recent generalist segment anything model) and further introduce a complementary architecture to better utilize underwater image enhancement for improving the segmentation performance of models. We have conducted extensive experiments based on various up-to-date segmentation models on our benchmark and the experimental results demonstrate that there is still ample room to improve coral segmentation performance. Ablation studies and discussions are also included. The proposed benchmark could significantly enhance the efficiency and accuracy of real-world underwater coral reef surveying.