Multi-stage image-based approach for fish detection and weight estimation

Challenges with sustainable use of aquatic resources stimulated the implementation of fishing regulations. To check compliance with regulations, observer programmes and electronic monitoring have been implemented but these suffer from low coverage because of extensive fishing activities and their high human-labour dependency. Aiming at automatic registration of the counts and weight per species in the discards, this work introduces a flexible image-based multi-stage approach composed by three stages: detection, classification, and weight estimation. Unlike single-stage approaches that require a single dataset containing the detection, classification, and weight information to train the model, the modular structure of the proposed approach allows training each component in an independent manner requiring only specific data for each stage (bounding boxes, species or weight), therefore different training sets could be used which is expected to improve overall fish detection and weight estimation. In the multi-stage approaches, the impact of using a general species-agnostic regressor vs species-specific regressors was also assessed. Experimental results on the Fish Detection and Weight Estimation dataset, containing 1086 images and 2216 fish instances, demonstrated the superiority of the proposed multi-stage approach over two single-stage methods. The localisation and classification tasks contributed to achieving an F1-macro of 92.72 %, surpassing the best single-stage approach by at least 6.41 percentage points. On the other hand, the localisation and regression tasks led to a MAPE-macro of 18.06, reducing the MAPE of the best single-stage approach by approximately 60 %. • A multi-stage approach for fish detection and weight estimation is proposed. • Task separation improves overall performance in fish detection and weight estimation. • The multi-stage approach outperformed the single-stage by 6.41 points on F1-macro. • The multi-stage approach reduced the single-stage MAPE by approximately 60 %. • Performance drops significantly in scenarios where fish are occluded by more than 60 %.