Evidence-Based Practice Policy Analysis and Program Evaluation Reporting Guidelines

INTRODUCTION In clinical practice, nurses may lead or participate in evidence-based practice quality improvement (EBPQI) initiatives in which an evidence-based practice change is implemented and data are continuously evaluated for improvement. While perhaps not as common, nurses may also be involved in robust evidence-based practice program evaluation (EBPPE) and policy analysis (EBPPA) initiatives. These types of initiatives are desperately needed to improve the quality of care provided. EBPPE initiatives focus on evaluating current “programs,” which broadly means any program, project, initiative, clinical pathway, and/or guideline. Often in health care, different programs or practices may be implemented but not systematically evaluated for effectiveness, such as a fall prevention program, nurse resource role, or nurse residency program. Additionally, EBPPA initiatives focus on evaluating policies for effectiveness, which could occur at the organizational or local level, as well as on the state and federal level. For example, nurses may evaluate an organizational policy on radiation safety practices to determine if the policy aligns with current evidence and to understand adherence to the policy. At a state level, nurses may evaluate a proposal for full scope of practice for advanced practice registered nurses’ policy and advocate for broad change.1 As with EBPQI initiatives, robust methods must be used to appropriately conduct EBPPE and EBPPA initiatives, which can be guided by the Mountain Model of EBPQI.2 The Mountain Model for EBPQI integrates EBP and QI in a seamless way to address deficiencies in each paradigm. EBP models lack strong implementation, evaluation, and sustainability components and QI models are often not built on a research or evidence foundation.2 The Mountain Model is also flexible, as an umbrella framework, and can be used with a variety of EBP and QI methods. To disseminate this work so other nurses can learn from, build on, and conduct similar EBPPE and EBPPA initiatives, there is a growing need for accurate reporting. Reporting guidelines are checklists that can provide guidance to authors as to the necessary components needed to write a comprehensive report of an initiative. Although there are multiple guidelines available for the reporting of research studies as found through the EQUATOR network, there is only one guideline available for potential use in reporting program evaluation initiatives3 that focuses on reporting use of research methods rather than using evaluation methods (value determination).4 Furthermore, no current reporting guidelines exist to guide reporting on policy analysis initiatives. This article introduces 2 additional guidelines for reporting EBPPE and EBPPA initiatives. These reporting guides serve as a companion guide to the recently published EBPQI reporting guideline5 framed by the Mountain Model umbrella framework.2 METHODS Following the methods used to develop the first EBPQI reporting guideline,5 and because of the scarcity of reporting guides, a literature search was conducted. PubMed, the Cumulative Index to Nursing and Allied Health Literature, and Google Scholar were searched using the terms program evaluation and reporting guidelines and policy analysis and reporting guidelines. Only peer-reviewed articles published in English were reviewed. No year limits were included in the search. Results of the literature search identified 2 articles that discussed and provided suggested criteria for reporting of program evaluations but were not formal reporting guides6,7; no articles were found regarding criteria for reporting policy analysis. The authors of the articles on program evaluation strongly remind readers that the purpose of program evaluation is not the same as research. The purpose of research is to generate new knowledge, whereas the purpose of program evaluation is to determine the value of the program (or practice).6,7 The criteria for evaluation may be variable depending on who the evaluator is (eg, funders, participants, communities, or those who are conducting the program or performing the practice)7; however, neither provides a comprehensive guideline for reporting an evidence-based practice program evaluation in health care. Therefore, the EBPPE and EBPPA reporting guidelines were developed based on (1) the gaps identified in the review of current reporting guidelines and the literature, (2) the Mountain Model of EBPPE and EBPPA,1 and (3) the currently available EBPQI reporting guideline.5 Each criterion is adapted to align with the specific methods of implementation for program evaluation and policy analysis. As with the EBPQI reporting guideline, the IMRaD (Introduction, Methods, Results, Discussion) structure was used.5 Table. - EBPPE AND EBPPA Reporting Guidelines (Combined) Component Description 1. Abstract The abstract summarizes the main points of the manuscript, including a statement that an EBPPE or EBPPA initiative was conducted. 2. Introduction Identifies the problem to be addressed by the EBPPE or EBPPA initiative. 3. Background Includes a description of the general and local practice problem with supporting evidence (eg, epidemiological, economic, health). 4.0 External evidence review and evidence-based practice component 4.1 Evidence search strategy Includes an evidence searching question (PPCO, PICO), key terms, and databases searched. Does not require a PRISMA flowchart. 4.2 Critical appraisal and evidence synthesis Includes a narrative synthesis of the quality of the evidence based on findings from critical appraisal tools (eg, Critical Appraisal Skills Programme checklists, Center for Evidence-based Medicine, EBPQI critical appraisal tool). Also includes a narrative synthesis of the evidence that will be used for the EBP recommendation. The best quality and most recent evidence should be included. 4.3 Clinical expertise Includes a description of the clinical expertise that informed the EBP recommendation (ie, what experts were included on the team that helped develop the recommendation). 4.4 Patient/ Population values/preferences/ culture Includes a description of how the population values, preferences, and culture were considered when developing the EBP recommendation. 5. Internal Evidence Internal evidence is included (may be in the background section) to document that there is a local problem and may be used as the baseline for improvement measures. 6. Purpose and Aims Includes a clear purpose statement and aims written using the SMART criteria (specific, measurable, achievable, relevant, time-bound). Aims are aligned with the method or model chosen. With policy analyses your aims may be more exploratory and descriptive in nature and less time-bound although they should be measurable. 7.0 Methods 7.1 Framework or Model A framework or model to guide the EBPPE or EBPPA is used. For example, the Logic Model or Kirkpatrick Model for EBPPE and Kingdon’s Multiple Streams Model or CDC Policy Analytic Framework for EBPPA. 7.2 Context and Population The local environment (context) where the program evaluation or policy analysis was/should be implemented is described. The description of the population to which the program or policies applies is stated. 7.3 Engagement The parties/individuals involved in the evaluation or policy analysis (stakeholders) are described. 7.4 Ethical Consideration Addresses ethical components of programs and policies. If appropriate, a statement about confidentiality may be included. 7.5. Specific application of the model or framework chosen Details of how the model or framework was applied to the EBPPE or EBPPA are described. 7.6 Measures and Analysis Aims are aligned with the method framework or model chosen. For each specific aim, include the operational definition (eg, what data were collected, how data collection occurred). Include how data were analyzed. 7.7 Results The results section addresses each stated specific aim. 8. Discussion Discussion includes a summary of the results and comparison to the synthesis of previous evidence. The discussion section should also address value determination of the program or policy analyzed. 9. Limitations Limitations address unavoidable and unpredicted occurrences and lessons learned. 10. Conclusion Frame results and implications for future practice and continued improvement. EBPPE, evidence-based practice program evaluation; EBPPA, evidence-based practice policy analysis; CDC, Centers for Disease Control and Prevention RESULTS The resulting EBPPE and EBPPA reporting guidelines include 10 categories and 19 criteria. The program evaluation (EBPPE) reporting guideline can be used to guide manuscript writing for program evaluation initiatives that may be submitted to peer-reviewed scholarly journals. The policy analysis (EBPPA) reporting guideline can be used to guide reports of policy analysis initiatives, which may include executive summaries, editorials, policy fact sheets, or manuscripts that may be submitted to peer-reviewed scholarly journals. While many policy analyses occur at the national level, these guidelines can also be used to guide reporting of local or institutional policy analysis. The EBPPE and EBPPA reporting guideline criteria with description are combined in the Table. Examples of how to apply the guidelines to both EBPPE and EBPPA initiatives can be found in Supplemental Digital Content, Table 1, Available at: https://links.lww.com/JNCQ/B485 Blank tables for use with EBPPE and EBPPA can be found in Supplemental Digital Content, Table 2, Available at: https://links.lww.com/JNCQ/B486 and Supplemental Digital Content, Table 3, Available at: https://links.lww.com/JNCQ/B493, respectively. IMPLICATIONS FOR NURSING CARE QUALITY In addition to leading EBPQI initiatives, nurses are well-positioned to also lead robust EBPPE and EBPPA initiatives to help improve the provision of quality nursing care. Programs, including practice changes from projects or initiatives, clinical pathways, and/or guidelines, need to be routinely evaluated to ensure they are meeting their intended outcomes. Policies at all levels (organizational, local, state, federal) also need to be analyzed to ensure they are up to date with current evidence and evaluate how they are being implemented and used. Findings from EBPPE and EBPPA initiatives, such as recommendations for a practice change, can be used to lead thoughtful EBPQI initiatives that are grounded in scientific evidence as well as local data. As EBPPE and EBPPA initiatives gain traction, comprehensive reporting guidelines are needed so clinicians can appropriately include necessary components during dissemination. Disseminating these important results from EBPPA and EBPPE initiatives can assist other nurses in leading similar projects within their own institutions, leading to broad improvements in care quality. CONCLUSIONS These 2 new reporting guidelines build on the available evidence and provide information on how to systematically and accurately report EBPPE and EBPPA initiatives. Conducting these types of initiatives that are grounded in the evidence and systematically conducted is foundational to the integrity of disseminated work and the future ability of nurses to transfer and adapt programs and policies to their own context. Using these reporting guidelines can help nurses share their work in a consistent and comprehensive manner.