Designing and validating entrustable professional activities for emergency medicine: a stringent, construct-validity enhancing approach

Entrustable professional activities (EPAs) are a cornerstone in many postgraduate medical education programs. However, the design of EPAs varies widely across programs, the validity of newly developed EPAs is rarely tested, and there is little agreement on how to validate them. This study aimed to describe and evaluate the consensus finding and validation process for the development of EPAs in Emergency Medicine (EM) in Switzerland following a stringent, construct-validity-enhancing approach. A conceptual framework was adopted as a template for the development and validation process. In the first step, a multi-institutional, core expert panel of EM physicians was assembled and received preparatory training. This group subsequently identified, developed, and revised EPAs in iterative rounds using Delphi and nominal group techniques. In the final step, a second expert panel, which was not involved in the development process, carried out validations of the EPAs using EQual rubric scores, generalizability analysis, and intra-class correlation coefficients. The development process resulted in 24 EPAs subdivided into 7 junior EPAs and 17 senior EPAs. The validation of the EPAs showed a mean total EQual rubric score of 4.08 (± 0.04) on a 5-point scale for all EPAs. The generalizability analysis yielded a phi-coefficient of 0.66 and an intra-class correlation coefficient of 0.83. By using and challenging existing strategies and validity-enhancing instruments, we were able to develop and validate EPAs in EM. Key elements in this process were the adherence to a conceptual framework as a structural planning template, multimodal preparatory education of experts, and validation by a second, independent expert group via appropriate tools such as the EQual rubric score.