Assessment of inter and intraobserver variability in manually delineated oral canine acanthomatous ameloblastoma from computed tomography scans

Radiomics involves quantitative analyses over specified regions to extract datapoints and quantitative features which can be used to train artificial intelligence models. Assessment of observer variability is performed to ensure reliability and reproducibility in the extracted quantitative features. Our primary objective is to compare the interobserver and intraobserver coefficients between participants in contouring computed tomographic images using canine acanthomatous ameloblastoma as a representative model. Ten canine acanthomatous ameloblastoma cases were randomly selected from Cornell University Hospital for Animals’ computed tomography database from 2014 to 2021 based on tumor visibility and availability of non-contrast and contrast enhanced images with bone and soft tissue reconstructions. These cases were evaluated and manually contoured in 3DSlicer by three specialists across two randomized sittings with instructions provided to guide contouring. Interobserver and intraobserver correlation coefficients for centroid coordinates and volumes indicated excellent agreement. Intraobserver Dice Similarity Index (DI) had moderate agreement (mean = 0.78±0.11) while interobserver DI was poor (mean = 0.69±0.14). Intraobserver mean Hausdorff Distances at 0.80±0.48 mm and interobserver mean Hausdorff Distances at 1.21±0.66 mm, indicating variability in contour comparisons between participants. Interobserver volume measurements were statistically different (P < 0.001), with moderate effect sizes. The excellent agreement in centroid coordinates and volumes supports consistent and reliable target region definition. However, there were statistically significant variabilities within the contour comparisons. The DI and Hausdorff Distances had overall moderate agreement; however, there was poor agreement with interparticipant DI, indicating less consistency between participants. The observed variability underscores the need for standardized contouring models to enhance reproducibility and reliability for clinical integration.