Weakly Supervised Segmentation of Macroalgae Through Gradient Analysis in Convolutional Neural Networks and Segment Anything Model

Segmenting marine macroalgae from optical imagery typically requires pixel-level annotations that are costly and hard to scale. This work proposes a weakly supervised pipeline that derives segmentation masks from image-level classification labels alone, eliminating manual spatial annotations. The method has three stages: (1) training a six-layer convolutional classifier from scratch, without weight transfer, on 1818 images of three macroalgae genera (Fucus, Laminaria, and Ulva); (2) analyzing the gradients of the classifier’s output with respect to all intermediate convolutional layers to build spatial heatmaps of the most decision-relevant regions; and (3) extracting keypoints via non-maximum suppression and using them as prompts for Segment Anything Model 2 (SAM2). We compare two importance metrics—sensitivity (gradient only) and contribution (gradient times activation)—and show that the former distributes importance more evenly across layers, preserving the resolution of early layers and yielding more precise localizations. The classifier reaches 94.5% accuracy with under 15 min of training on conventional hardware. On 42 manually annotated validation images used exclusively for evaluation, the complete pipeline achieves a mean IoU of 0.44, Dice coefficient of 0.51, precision of 0.70, and recall of 0.54, while 80% of the extracted keypoints fall inside the annotated algae.