Where the error comes from in automated muscle architecture measurement: a case study on the UMUD Kaggle challenge
The UMUD Kaggle challenge asks for pennation angle (PA), fascicle length (FL) and muscle thickness (MT) from a single B-mode ultrasound image. The training data contain segmentation masks and no measurements, so most solutions, ours included, segment aponeuroses and fascicles with U-Nets and compute the three numbers with hand-written geometry. We describe such a pipeline and analyse where its error comes from. The main findings are four. (1) The competition score is governed by the error in PA, because FL inherits it: at a typical vastus lateralis angle of 12°, one degree of PA error moves FL by about 9 mm. Per frame, our score is close to 0.13 + 0.14·|e_PA|. (2) A structure tensor computed on the raw image underestimates PA on all 35 images of the UMUD expert benchmark, by 5.6° on average and more for steeper fascicles; a two-term tensor model with laterally elongated speckle explains the trend. Computing orientation on the network's fascicle probability map reduces the bias to −0.8°. (3) On the benchmark, systematic differences between seven expert raters (PA −1.0° to +1.3°, FL −5.7 to +6.2 mm) are as large as the error of a good model, so matching the test raters' convention matters more than segmentation accuracy. (4) Manual measurements of the test images by one non-expert team member improved the public score from 0.441 to 0.385 when used as targets for a cross-fitted residual model, scored 0.458 when submitted directly, and misled local validation when a network was fine-tuned on the clicks themselves. Every submitted value is computed by the pipeline from the image; the public leaderboard influenced four frozen constants and the choice of the final variant, all listed in the paper. Our best public score is 0.383. Code: https://github.com/Aldabergenov1/umud-challenge
Authors
- Arman Zhalgasbayev (ORCID: https://orcid.org/0000-0001-8507-6845)
- Makhambet Aldabergenov
Institutions
- Kazakh-British Technical University (KZ)
- Narxoz University (KZ)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-17
- DOI
- https://doi.org/10.5281/zenodo.24321877
- Primary Topic
- Medical Image Segmentation Techniques
- Type
- preprint