Development and validation of the Omentectomy Assessment Tool Score (OATS) - a novel intraoperative performance assessment tool for surgical education

The global burden of non-communicable diseases (NCD), including cancer, continues to rise worldwide; currently 30% of NCD deaths are attributable to cancer. The omentum, a protective peritoneal layer, is frequently involved in gynecological malignancies, and omentectomy is therefore a common procedure in their surgical management. Complete surgical procedures, including omentectomy, are associated with better survival outcomes. Despite its importance, no standardized tool exists for assessing this essential competence in gynecologists. To develop and validate a tool with reliable scores for Omentectomy to assess the OBGYN trainees in acquiring the skills of performing Omentectomy. This was a pilot study conducted in the OBGYN department of Aga Khan University. After developing the Omentectomy Assessment Tool Score (OATS), simulation-based hands-on workshops were held at AKUH and JPMC. Forty-three OBGYN trainees were assessed on OATS by two raters simultaneously but independently, using a seven-point adjectival scale (0-6). Messick’s unitary validity framework was used to establish validity evidence. Inter-rater reliability was computed by intraclass correlation (ICC) and internal consistency by Cronbach’s alpha. Discriminant power between resident years was planned via Kruskal-Wallis’s test, but due to small numbers per year, residents were collapsed into one group and compared with fellows using the Mann-Whitney U test. p<0.05 was taken as significant. Cronbach’s alpha of the 18 items of OATS tool was 0.94. ICC of 13 out of 18 items range from 0.75 to 0.84, while 5 of 18 items range of 0.64 to 0.74. However, none of the ICC value was below 0.5. There is no significant difference in the mean scores of different levels of trainees. This limits the evidence for discriminant (known-groups) validity of the tool; both groups scored near the mid-point of the 0-6 scale, which may reflect the small sample size and/or limitations of the simulation model's fidelity. This pilot study provides preliminary evidence for the reliability of OATS in a simulation-based setting; however, discriminant validity across training levels was not established, and its use for credentialing would be premature pending larger, multicenter validation in live operative settings.

Authors

Institutions

Publication Details

Journal
BMC Medical Education
Published
2026-11-16
DOI
https://doi.org/10.1186/s12909-026-10649-1
Primary Topic
Surgical Simulation and Training
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Development and validation of the Omentectomy Assessment Tool Score (OATS) - a novel intraoperative performance assessment tool for surgical education

Uzma Chishti, Aliya Begum Aziz, Amir Raza, Azam Saeed Afzal et al.
1 links
BMC Medical Education
Surgical Simulation and Training
article

Development and validation of the Omentectomy Assessment Tool Score (OATS) - a novel intraoperative performance assessment tool for surgical education

Uzma Chishti, Aliya Begum Aziz, Amir Raza, Azam Saeed Afzal, Munazza Akhtar, Shazia Babar
article en

Abstract

The global burden of non-communicable diseases (NCD), including cancer, continues to rise worldwide; currently 30% of NCD deaths are attributable to cancer. The omentum, a protective peritoneal layer, is frequently involved in gynecological malignancies, and omentectomy is therefore a common procedure in their surgical management. Complete surgical procedures, including omentectomy, are associated with better survival outcomes. Despite its importance, no standardized tool exists for assessing this essential competence in gynecologists. To develop and validate a tool with reliable scores for Omentectomy to assess the OBGYN trainees in acquiring the skills of performing Omentectomy. This was a pilot study conducted in the OBGYN department of Aga Khan University. After developing the Omentectomy Assessment Tool Score (OATS), simulation-based hands-on workshops were held at AKUH and JPMC. Forty-three OBGYN trainees were assessed on OATS by two raters simultaneously but independently, using a seven-point adjectival scale (0-6). Messick’s unitary validity framework was used to establish validity evidence. Inter-rater reliability was computed by intraclass correlation (ICC) and internal consistency by Cronbach’s alpha. Discriminant power between resident years was planned via Kruskal-Wallis’s test, but due to small numbers per year, residents were collapsed into one group and compared with fellows using the Mann-Whitney U test. p<0.05 was taken as significant. Cronbach’s alpha of the 18 items of OATS tool was 0.94. ICC of 13 out of 18 items range from 0.75 to 0.84, while 5 of 18 items range of 0.64 to 0.74. However, none of the ICC value was below 0.5. There is no significant difference in the mean scores of different levels of trainees. This limits the evidence for discriminant (known-groups) validity of the tool; both groups scored near the mid-point of the 0-6 scale, which may reflect the small sample size and/or limitations of the simulation model's fidelity. This pilot study provides preliminary evidence for the reliability of OATS in a simulation-based setting; however, discriminant validity across training levels was not established, and its use for credentialing would be premature pending larger, multicenter validation in live operative settings.

BMC Medical Education
Aga Khan University (PK), Aga Khan University Hospital (PK)
Good health and well-being
Surgical Simulation and Training
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon