Reliability of a Progressive Learning Autonomy Assessment in a Periodontology Residency Program

PURPOSE: This study aimed to evaluate the reliability of a surgical progressive learning autonomy assessment tool implemented in a postgraduate periodontology residency program. METHODS: A prospective observational study was conducted during the Spring and Summer 2025 semesters using a structured, real-time surgical assessment tool. Faculty assessed resident performance after each surgical procedure across domains of autonomy, performance, and case complexity on a five-point scale, while residents completed parallel self-assessments for the same procedures. Inter-rater reliability among faculty and agreement between faculty and residents were evaluated using intraclass correlation coefficients (ICCs). Mixed-model ANOVA was used to compare effects of resident year, procedure type, and rater group (faculty or resident) on mean scores. RESULTS: A total of 561 surgical assessments were analyzed. Faculty inter-rater reliability ranged from low to moderate, with the highest agreement observed for implant placement procedures (ICC = 0.58) and lower agreement for crown lengthening (ICC = 0.13) and extraction with ridge preservation (ICC = 0.09). Agreement between faculty and residents was stronger than inter-faculty agreement for most procedures (ICC > 0.3 for crown lengthening, guided bone regeneration, impant placement osseous). Faculty scores were consistently higher than resident self-scores (p < 0.05 for several procedure-question combinations by 0.3-0.5 points). Residents demonstrated significant progression in scores from Years 1 to 3 for selected procedures, particularly crown lengthening, soft tissue grafting, and extraction-related surgeries. Graduating residents demonstrated competency based on both faculty and self-assessment benchmarks. CONCLUSIONS: A progressive learning autonomy assessment provides reliable, meaningful data on resident development, and highlights differences between perceived and observed competence. This model supports real-time feedback, resident growth, and faculty calibration, with implications for broader adoption in dental specialty education.