Optimization and Benchmarking of Image Segmentation for Improved Landmark Detection in Lower Limb X-Rays and Accurate Coronal Plane Alignment of the Knee Classification

Recent studies have explored image segmentation for landmark detection in computer vision and medical imaging of the lower limb, showing promising results. However, proposed methodologies strongly vary, and comparison with existing methods is lacking. In the present work, we investigate image segmentation for landmark detection on full lower-limb X-rays in detail and benchmark it against conventional landmark detection approaches. We detected eight landmarks in full lower limb X-rays and investigated methodological aspects to optimize image segmentation performance: network architecture (U-Net vs. Swin-UNETR), the mask size centered at the landmark position to segment, and the coordinate computation technique from the segmentation map. We contrasted image segmentation against optimized heatmap, coordinate, and segmentation-guided coordinate regression methods. The evaluation assessed the landmark detection error and phenotype classification accuracy based on lower limb alignment. The optimal segmentation approach employed a U-Net to segment circular masks (radius = 15 pixels), using probability thresholding before centroid computation. Regarding landmark detection accuracy, image segmentation (median Euclidean distance (inter-quartile range) = 1.16 mm (1.50 mm)) was more accurate than heatmap (1.19 mm (1.61 mm)), coordinate (3.11 mm (2.87 mm)), and segmentation-guided coordinate regression (1.47 mm (1.67 mm)). Image segmentation outperformed heatmap, coordinate, and segmentation-guided coordinate regression in phenotype classification accuracy, achieving an average F<sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub>-score of 0.79, versus 0.72, 0.47, and 0.77, respectively. Our study led to an optimized approach for landmark detection using image segmentation, outperforming alternative detection approaches tuned and tested on the same data, highlighting image segmentation’s potential for broader medical imaging research applications.