A Community Benchmark for the Automated Segmentation of Pediatric Neuroblastoma on Multi-Modal MRI: Design and Results of the SPPIN Challenge at MICCAI 2023

Surgery plays a key role in treating neuroblastoma. To assist surgical planning, anatomical 3D models derived from the segmentation of anatomical structures on MRI scans are often used. Automation using deep learning can make segmentations less time-consuming and more reliable. We organized the Surgical Planning in PedIatric Neuroblastoma (SPPIN) challenge, to stimulate developments and benchmarking of automatic segmentation of neuroblastoma on MRI. SPPIN is the first segmentation challenge in extracranial pediatric oncology. Nine teams provided a valid submission. Evaluation was based on the Dice similarity coefficient (Dice score), the 95th percentile of the Hausdorff distance (HD95), and the volumetric similarity (VS). A combination of these scores determined the ranking of the teams. The spread in the median evaluation scores per team was large (Dice: 0.21-0.82; HD95: 63.31-7.69; VS: 0.31-0.91). The top-performing team achieved a median Dice score of 0.82 (with an HD95 of 7.69 mm and a VS of 0.91) using a large, pre-trained model. However, in the pre-operative segmentations, significantly lower evaluation scores were observed. Our results indicate that pre-training might be useful in small, pediatric datasets. Although the general results of the winning team were high, they were insufficient to use for surgical planning in small, pre-operative tumors.

Authors

Institutions

Publication Details

Journal
Bioengineering
Published
2025-10-26
DOI
https://doi.org/10.3390/bioengineering12111157
Primary Topic
Neuroblastoma Research and Treatments
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

A Community Benchmark for the Automated Segmentation of Pediatric Neuroblastoma on Multi-Modal MRI: Design and Results of the SPPIN Challenge at MICCAI 2023

Maximilian Rokuss, Matthijs Fitski, Iris N. Vos, Diana Veiga-Canuto et al.
Bioengineering
Neuroblastoma Research and Treatments
article

A Community Benchmark for the Automated Segmentation of Pediatric Neuroblastoma on Multi-Modal MRI: Design and Results of the SPPIN Challenge at MICCAI 2023

Maximilian Rokuss, Matthijs Fitski, Iris N. Vos, Diana Veiga-Canuto, Shoji Kido, Alida F. W. van der Steeg, Marek Wodziński, David Ellis, Matías Fernández-Patón, Bas H. M. van der Velden, Alberto De Luca, Junya Sato, Markus H. A. Janse, Marc H. W. A. Wijnen, Annemieke Simone Littooij, Junjun He, Mathijs de Boer, Henning Müller, Satoshi Kondo, Dominique C. Simons, Hugo J. Kuijf, Michele R. Aizenberg, Myrthe A. D. Buser, Yannick Kirchhoff, Satoshi Kasai, Rens ter Maat, Gao Haokai, Annemiek H. ter Brugge
article en

Abstract

Surgery plays a key role in treating neuroblastoma. To assist surgical planning, anatomical 3D models derived from the segmentation of anatomical structures on MRI scans are often used. Automation using deep learning can make segmentations less time-consuming and more reliable. We organized the Surgical Planning in PedIatric Neuroblastoma (SPPIN) challenge, to stimulate developments and benchmarking of automatic segmentation of neuroblastoma on MRI. SPPIN is the first segmentation challenge in extracranial pediatric oncology. Nine teams provided a valid submission. Evaluation was based on the Dice similarity coefficient (Dice score), the 95th percentile of the Hausdorff distance (HD95), and the volumetric similarity (VS). A combination of these scores determined the ranking of the teams. The spread in the median evaluation scores per team was large (Dice: 0.21-0.82; HD95: 63.31-7.69; VS: 0.31-0.91). The top-performing team achieved a median Dice score of 0.82 (with an HD95 of 7.69 mm and a VS of 0.91) using a large, pre-trained model. However, in the pre-operative segmentations, significantly lower evaluation scores were observed. Our results indicate that pre-training might be useful in small, pediatric datasets. Although the general results of the winning team were high, they were insufficient to use for surgical planning in small, pre-operative tumors.

BioengineeringVol. 12(11)
University of Geneva (CH), Fujita Health University (JP), HES-SO University of Applied Sciences and Arts Western Switzerland (CH), German Cancer Research Center (DE), South China Normal University (CN), Utrecht University (NL), University School of Physical Education in Kraków (PL), University Medical Center Utrecht (NL), University of Agriculture in Krakow (PL), Muroran Institute of Technology (JP), Beijing Academy of Artificial Intelligence (CN), University Hospital of Geneva (CH), Leitat Technological Center (ES), Princess Máxima Center (NL), Instituto de Investigación Sanitaria La Fe (ES), Shanghai Artificial Intelligence Laboratory (CN), University of Nebraska Medical Center (US), Wageningen University & Research (NL), The University of Osaka (JP)
Openalex Percentile: Top 56%
Neuroblastoma Research and Treatments
DOIOpen Access PDFOpenAlex
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.