Consensus-Based Marking Schemes to Assess Competence Using Free-Text Echocardiogram Summary Statements

BackgroundSummary statements of transthoracic echocardiography (TTE) reports can be used to assess trainees' competence in TTE interpretation.However, the inherent variability in wording and composition of summary statements creates challenges for developing marking schemes.This study aims to develop consensus-based marking schemes for TTE summary statements. MethodsA Modified Delphi approach with two rounds of surveys and a consensus finalizing meeting was conducted to establish items that should be included in the summary statement and items critical to fulfill competence.Via consensus (defined as >80% agreement), experts determined what items should always be included, and what items should be included in specific indications or cardiac diagnoses. ResultsThe first and second round surveys were completed by 27 experts.Eleven experts participated in the consensus finalizing meeting.Through this process, it was established that 79 items should be included in the summary statement.Five general items (left ventricular (LV) size, LV systolic function, right ventricular (RV) size and RV function, and a statement about valves) should be included in all summary statements, while the remaining 74 items should be included in the presence of abnormalities, specific indications or relevant diagnoses.Sixteen items were deemed critical for J o u r n a l P r e -p r o o f competence.Only LV systolic function was deemed critical to fulfill competence in all summary statements, with the remaining 15 items critical in specific indications. ConclusionsThese findings inform consensus-based marking schemes for TTE summary statements.When integrated into competence assessments, these findings will inform the training of TTE interpretation for cardiology residents.