Leveraging the power of diagnostic metrics to competency based medical education (CBME) implementation in medical oncology (MO) across Canada.

9025 Background: MO training programs across Canada implemented CBME in 2018. Early implementation focused primarily on immediate “structural” changes, and included the adoption of new stages of training, new assessment practices and the creation of Competence Committees. To explore elements that would reflect broader and transformational change related to a true shift to an individualized and competency-based approach to education, program leaders sought to identify, develop, and pilot indicators that could be used by programs to evaluate their implementation of the Competence by Design (CBD) model. It was anticipated that implementation and evaluation of these indicators would be challenging. Methods: In phases one and two of the study, program leaders established a consensus regarding qualities they considered to transformative, and qualitative information regarding how these qualities are reflected in programs was obtained. In phase three, electronic resident portfolios at 2 sample sites were investigated for data regarding specific indicators to determine the feasibility of use by program directors to track implementation progress and aid in program review. Opinions of program leaders in all 14 Canadian programs were obtained through a consensus process. Educators from all sites were invited to participate in semi-structured interviews and a 100% response rate obtained. Data from the 2 sample sites was collected from portfolios, de-identified and reported in aggregate to help maintain confidentiality. Results: 7 key priority indicators were identified. These centered around 6 themes: direct observation, personal learning plans, curricular change, coaching, data sources used by Competency Committees and general concerns about CBD. Variability was found in the extent of implementation of these across programs and in adaptations made locally. At the 2 sample sites, extraction of key metrical indicators from resident portfolios had to be completed manually and was challenging as electronic databases had not been designed to allow easy review and analysis of these specific indicators. Conclusions: Program leaders of Canadian MO training programs were able to reach consensus regarding key data indicators they believe to be transformative and reflective of core CBD principles. Despite this consensus, variability was found in the implementation of these across programs and practical challenges encountered in extracting data related to key indicators from resident portfolios at 2 sample sites. To provide program leaders with data they feel is important for optimal CBD implementation, electronic databases will need ongoing attention and adaptation to facilitate access to key indicators considered important for program review and evaluation.