Using a Consensus Reference Standard to Achieve High Reliability in Grading Aortic Stenosis in an Academic Pediatric Echo Lab

Accuracy and reliability are important in grading aortic stenosis (AS) by echocardiogram. There are no pediatric-specific consensus guidelines for grading AS by echo. Our objective was to adopt a consensus reference standard for AS and evaluate resulting interrater reliability in a pediatric echo laboratory. A reference standard for grading AS was derived from published adult guidelines. Measurements for mean gradient and aortic valve area as determined by the continuity equation were included. Comparisons were made between AS grades on 20 historic echo reports and AS grades using the new reference standard. Additionally, 6 readers used the reference standard to grade 10 studies prospectively and interrater reliability was evaluated for measurements required for grading AS and overall AS grade. Twenty echo studies were included (median subject age 7 IQR 4-16 years, median weight 32.2 IQR 13.8-44.8 kg). AS grade on historic echo reports and AS grade using new reference standard were in agreement 85% of the time. In the prospective portion of the study there was substantial agreement between raters on overall AS grade (Fleiss' kappa statistic 0.76 95% CI 0.54-0.98). There was excellent interrater reliability in measures of peak gradient (ICC 0.92, 95% CI 0.82-0.98), mean gradient (ICC 0.95, 95% CI 0.88-0.98), left ventricular outflow tract diameter (ICC 0.96, 95% CI 0.92-0.99), and left ventricular outflow tract velocity time integral (ICC 0.94, 95% CI 0.75-0.99). Using a consensus reference standard for grading AS may result in high interrater reliability in pediatric echo labs. Measurements of the mean gradient and measurements needed for the continuity equation for aortic valve area have high interrater reliability.