Enhancing Landmark Point Detection in Eriocheir Sinensis Carapace with Differentiable End-to-End Networks

This research proposes using a neural network to detect and identify the landmark points of the carapace of the Chinese mitten crab, with the aim of improving efficiency in observation, measurement, and statistics in breeding and sales. A 37-point localization framework was developed for the carapace, with the dataset augmented through random distortions, rotations, and occlusions to enhance generalization capability. Three types of convolutional neural network models were used to compare detection accuracy, generalization ability, and model power consumption, with different loss functions compared. The results showed that the Convolutional Neural Network (CNN) model based on the Differentiable Spatial to Numerical Transform (DSNT) module had the highest R² value of 0.9906 on the test set, followed by the CNN model based on the Gaussian heatmap at 0.9846. The DSNT-based CNN model exhibited optimal computational efficiency, particularly in power consumption metrics. This research demonstrates that the CNN model based on the DSNT module has great potential in detecting landmark points for the Chinese mitten crab, reducing manual workload in breeding observation and quality inspection, and improving efficiency.