Where the error comes from in automated muscle architecture measurement: a case study on the UMUD Kaggle challenge

The UMUD Kaggle challenge asks for pennation angle (PA), fascicle length (FL) and muscle thickness (MT) from a single B-mode ultrasound image. The training data contain segmentation masks and no measurements, so most solutions, ours included, segment aponeuroses and fascicles with U-Nets and compute the three numbers with hand-written geometry. We describe such a pipeline and analyse where its error comes from. The main findings are four. (1) The competition score is governed by the error in PA, because FL inherits it: at a typical vastus lateralis angle of 12°, one degree of PA error moves FL by about 9 mm. Per frame, our score is close to 0.13 + 0.14·|e_PA|. (2) A structure tensor computed on the raw image underestimates PA on all 35 images of the UMUD expert benchmark, by 5.6° on average and more for steeper fascicles; a two-term tensor model with laterally elongated speckle explains the trend. Computing orientation on the network's fascicle probability map reduces the bias to −0.8°. (3) On the benchmark, systematic differences between seven expert raters (PA −1.0° to +1.3°, FL −5.7 to +6.2 mm) are as large as the error of a good model, so matching the test raters' convention matters more than segmentation accuracy. (4) Manual measurements of the test images by one non-expert team member improved the public score from 0.441 to 0.385 when used as targets for a cross-fitted residual model, scored 0.458 when submitted directly, and misled local validation when a network was fine-tuned on the clicks themselves. Every submitted value is computed by the pipeline from the image; the public leaderboard influenced four frozen constants and the choice of the final variant, all listed in the paper. Our best public score is 0.383. Code: https://github.com/Aldabergenov1/umud-challenge

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-17
DOI
https://doi.org/10.5281/zenodo.24321877
Primary Topic
Medical Image Segmentation Techniques
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Where the error comes from in automated muscle architecture measurement: a case study on the UMUD Kaggle challenge

Arman Zhalgasbayev, Makhambet Aldabergenov
1 links
Zenodo (CERN European Organization for Nuclear Research)
Medical Image Segmentation Techniques
preprint

Where the error comes from in automated muscle architecture measurement: a case study on the UMUD Kaggle challenge

Arman Zhalgasbayev, Makhambet Aldabergenov
preprint en

Abstract

The UMUD Kaggle challenge asks for pennation angle (PA), fascicle length (FL) and muscle thickness (MT) from a single B-mode ultrasound image. The training data contain segmentation masks and no measurements, so most solutions, ours included, segment aponeuroses and fascicles with U-Nets and compute the three numbers with hand-written geometry. We describe such a pipeline and analyse where its error comes from. The main findings are four. (1) The competition score is governed by the error in PA, because FL inherits it: at a typical vastus lateralis angle of 12°, one degree of PA error moves FL by about 9 mm. Per frame, our score is close to 0.13 + 0.14·|e_PA|. (2) A structure tensor computed on the raw image underestimates PA on all 35 images of the UMUD expert benchmark, by 5.6° on average and more for steeper fascicles; a two-term tensor model with laterally elongated speckle explains the trend. Computing orientation on the network's fascicle probability map reduces the bias to −0.8°. (3) On the benchmark, systematic differences between seven expert raters (PA −1.0° to +1.3°, FL −5.7 to +6.2 mm) are as large as the error of a good model, so matching the test raters' convention matters more than segmentation accuracy. (4) Manual measurements of the test images by one non-expert team member improved the public score from 0.441 to 0.385 when used as targets for a cross-fitted residual model, scored 0.458 when submitted directly, and misled local validation when a network was fine-tuned on the clicks themselves. Every submitted value is computed by the pipeline from the image; the public leaderboard influenced four frozen constants and the choice of the final variant, all listed in the paper. Our best public score is 0.383. Code: https://github.com/Aldabergenov1/umud-challenge

Zenodo (CERN European Organization for Nuclear Research)
Kazakh-British Technical University (KZ), Narxoz University (KZ)
Medical Image Segmentation Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon