Inter-Rater Reliability Of Assigning A Pediatric Appendicitis Grade: An International Multicenter Study By The Canadian Consortium For Research In Pediatric Surgery (CanCORPS)

Background: Appendicitis is typically described, treated, and reported in outcome studies as simple versus perforated, without appropriate risk stratification. Objective: We performed an international study to assess the inter-rater reliability of assigning a Pediatric Appendicitis Grade (PAG) to children undergoing appendectomy. Methods: Pediatric surgery attending staff and trainees from 29 institutions in 8 countries on 6 continents rated 15 edited videos of laparoscopic appendectomy (3 for each PAG) on a web-based platform. Error rates were compared using Fisher exact test. Inter-observer reliability was estimated with the intraclass correlation coefficient (ICC), computed in R with a 2-way random-effects model, reported on a scale of 0 (no correlation) to 1 (perfect agreement) with a 95% confidence interval (CI). Results: Of 274 eligible participants, 121 respondents (44%) completed the study. The proportion of correctly identified cases was 71% overall: PAG 1, 97%; PAG 2, 70%; PAG 3, 73%; PAG 4, 64%; PAG 5, 51%. ICC was 0.81 (95% CI = 0.65–0.94), indicating good reliability among raters. There were no significant subgroup differences in ICC when comparing attending staff (ICC = 0.85, 95% CI = 0.74–0.94) versus trainees (0.84, 95% CI = 0.73–0.93), or when comparing respondents with low (0.84, 95% CI = 0.73–0.93), medium (0.81, 95% CI = 0.70–0.92), or high case volumes (0.80, 95% CI = 0.68–0.91). Conclusion: The PAG can be accurately and reliably assigned by surgeons and trainees in diverse settings and practices. Its routine use can improve prognostication, benchmarking, outcomes reporting, and resource allocation for the most common acute surgical illness in children.

Authors

Institutions

Publication Details

Journal
Annals of Surgery Open
Published
2026-04-16
DOI
https://doi.org/10.1097/as9.0000000000000657
Primary Topic
Appendicitis Diagnosis and Management
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Inter-Rater Reliability Of Assigning A Pediatric Appendicitis Grade: An International Multicenter Study By The Canadian Consortium For Research In Pediatric Surgery (CanCORPS)

Nelson Piché, Mehul V. Raval, Ahmed Nasr, Sameh Shehata et al.
Annals of Surgery Open
Appendicitis Diagnosis and Management
article

Inter-Rater Reliability Of Assigning A Pediatric Appendicitis Grade: An International Multicenter Study By The Canadian Consortium For Research In Pediatric Surgery (CanCORPS)

Nelson Piché, Mehul V. Raval, Ahmed Nasr, Sameh Shehata, Hayden W. Stagg, Sherif Emil, Augusto Zani, Fouad Youssef, Elena Guadagno, Pascale Prasil, A. Hodder, Andreana Bütter, Christopher Blackmore, Paul W. Wales, Yasmine Yousef, Michael Livingston, Martina Mudri, Ioana Bratu, Marcelo Eller Miranda, Alana L. Beres, Etienne St‐Louis, Natalie Yanchar, Sandeep Mayer, Mercedes Pilkington, F. P. Botelho, Amanda Hall, Nigel J. Hall, Ramon Gorter, Hebatallah Taher, Dan Poenaru, Aly Shalaby, Kartikey Pandya, Robert Baird, Gabriel Anthony Villa, Jonathan Wells
article en

Abstract

Background: Appendicitis is typically described, treated, and reported in outcome studies as simple versus perforated, without appropriate risk stratification. Objective: We performed an international study to assess the inter-rater reliability of assigning a Pediatric Appendicitis Grade (PAG) to children undergoing appendectomy. Methods: Pediatric surgery attending staff and trainees from 29 institutions in 8 countries on 6 continents rated 15 edited videos of laparoscopic appendectomy (3 for each PAG) on a web-based platform. Error rates were compared using Fisher exact test. Inter-observer reliability was estimated with the intraclass correlation coefficient (ICC), computed in R with a 2-way random-effects model, reported on a scale of 0 (no correlation) to 1 (perfect agreement) with a 95% confidence interval (CI). Results: Of 274 eligible participants, 121 respondents (44%) completed the study. The proportion of correctly identified cases was 71% overall: PAG 1, 97%; PAG 2, 70%; PAG 3, 73%; PAG 4, 64%; PAG 5, 51%. ICC was 0.81 (95% CI = 0.65–0.94), indicating good reliability among raters. There were no significant subgroup differences in ICC when comparing attending staff (ICC = 0.85, 95% CI = 0.74–0.94) versus trainees (0.84, 95% CI = 0.73–0.93), or when comparing respondents with low (0.84, 95% CI = 0.73–0.93), medium (0.81, 95% CI = 0.70–0.92), or high case volumes (0.80, 95% CI = 0.68–0.91). Conclusion: The PAG can be accurately and reliably assigned by surgeons and trainees in diverse settings and practices. Its routine use can improve prognostication, benchmarking, outcomes reporting, and resource allocation for the most common acute surgical illness in children.

Annals of Surgery OpenVol. 7(2)
Cincinnati Children's Hospital Medical Center (US), Lurie Children's Hospital (US), Cairo University (EG), Children's Hospital of Eastern Ontario (CA), Saskatoon City Hospital (CA), Montreal Children's Hospital (CA), Izaak Walton Killam Health Centre (CA), London Health Sciences Centre (CA), Hospital for Sick Children (CA), Janeway Children's Health and Rehabilitation Centre (CA), McGill University Health Centre (CA), Centre Hospitalier Universitaire de Sherbrooke (CA), McMaster Children's Hospital (CA), Alberta Children's Hospital (CA), Centre Hospitalier Universitaire Sainte-Justine (CA), Nova Scotia Health Authority (CA), Christchurch Hospital (NZ), Centre hospitalier universitaire de Québec (CA), Emma Kinderziekenhuis (NL), Memorial Healthcare System (US), Baylor Scott & White Medical Center - Temple (US), Cebu Doctors' University (PH), MaineHealth (US), Children's Hospital of Winnipeg (CA), Victoria General Hospital (CA), Hospital das Clínicas da Universidade Federal de Minas Gerais (BR), BC Children's Hospital (CA), Stollery Children's Hospital (CA), Joe DiMaggio Children's Hospital (US), St. Christopher's Hospital for Children (US), Children Cancer Hospital (EG), University of Southampton (GB), University of Otago (NZ), Alexandria University (EG)
Partnerships for the goals
Openalex Percentile: Top 19%
Appendicitis Diagnosis and Management
DOIOpen Access PDFOpenAlex
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.