Box2Rip: Instance Segmentation of Amorphous Rip Currents via Box-Supervised Learning

Rip currents are hazardous offshore water flows that significantly threaten swimmers and bathers, pulling them away from the shore at velocities up to 2.4 <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ms</i><sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">−1</sup>. Due to their amorphous structure and indistinct boundaries, accurately identifying rip currents remains a challenging task. While deep learning (DL)-based segmentation models have shown promise, they typically rely on laborious and resource-intensive pixel-level annotations. In this paper, we address this crucial problem by introducing Box2Rip, a novel box-supervised instance segmentation (BSIS) framework designed for rip current segmentation using only bounding box annotations. Box2Rip employs a two-phase architecture encompassing an extent-sensitive mask generator, mask refiner, and bbox matcher. Besides, we propose a joint loss system to encourage the predicted masks to adhere to the bounding boxes while capturing rip current boundaries. Our extensive experiments on the Dumitriu dataset with 2,466 images show that Box2Rip with ResNet-50 backbone achieves 76.2% <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">AP</i><sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">50:95</sub>, outperforming fully supervised models by 66.3% and exceeding box-supervised methods by 9.4%. On the AI-Hub CCTV rip current dataset containing 30,000 images, Box2Rip attained 38.6% <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">AP</i><sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">50:95</sub>, 76.2% <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">AP</i><sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">50</sub>, and 60.2% AR with minimal bounding box supervision. The proposed framework demonstrates exceptional capability in delineating amorphous rip currents with significantly reduced annotation effort, enabling efficient and scalable rip current detection systems that enhance coastal safety measures.