Obstacles and Difficulties in Conducting Systematic Reviews

INTRODUCTION A systematic review (SR) follows a structured method of gathering and analyzing papers, ensuring replicability, and presenting summarized findings.[1] This process typically involves collaborative teamwork. A protocol is a reference document that outlines a plan or roadmap, made explicit before the study is undertaken.[2] There are standard guidelines, one of which is commonly followed, called the Preferred Reporting Items for Systematic Reviews and Meta-Analyses.[3] Searches are conducted in pre-determined electronic databases. Authors take on various roles, progressing through the screening stage, where eligibility criteria are applied to select the final set of articles for review and data pooling. Meta-analysis is conducted and results are generated. There are several steps involved in the entire process, where obstacles are bound to appear and can bring the work to a halt altogether. Here, we discuss obstacles and difficulties during various stages and how they can be overcome. For simplicity, we can divide the stages into protocol registration, selection of papers for review, and analysis of data. FROM RESEARCH IDEA TO PROTOCOL A protocol is prepared to guide the SR process. PROSPERO is one of the most commonly used registries, which provides the necessary template. All mandatory sections must be completed.[4] The abundance of SRs makes it challenging for authors to identify a unique and specific topic for their review. Choosing a complex topic or niche area can create difficulties when selecting a research topic, especially for beginners. Opting for relatively simpler topics can make the research process more manageable. Due to unanticipated challenges, sometimes adjustments and deviations to the protocol may be necessary.[5] Although PROSPERO allows amendments, the approval can sometimes take longer, potentially delaying the process. Thus, significant effort should be made during the initial formulation of the protocol. When the individual study findings are negative, authors may feel discouraged. When selecting statistical methods, it is advisable to include alternative strategies in the protocol to allow flexibility during the analysis. Thus, the protocol stage involves lots of anticipatory thinking where emerging doubts and ambivalence may cause hurdles. SELECTION OF PAPERS Search results form the foundation of any SR, making each step crucial to its success. It is hard to return to this stage if one realizes errors in search strategy later. Comprehensive search strategies for identifying studies in the existing literature must be carefully crafted. Different databases have different search strategies, and authors must become familiar with the available databases. A very strict search strategy may overlook important studies, while broad strategies can produce an overwhelming number of papers. Authors should also familiarize themselves with relevant grey literature. Further, decision-making may be hindered by procrastination and the author needs to be mindful of it. Authors may face challenges if the eligibility criteria are not clearly defined from the outset. Initial screening based on titles and abstracts may seem straightforward, but titles do not always adequately reflect the research content. In such cases, a thorough abstract review is necessary before excluding an article. Occasionally, papers may not have content that matches the title and may not have results relevant to the SR of concern. Sometimes, the required data are incomplete or missing. In these cases, corresponding authors can be contacted for additional data, allowing the article to be included if the required information is obtained. Authors must ensure that any protocol deviations do not introduce bias. Writing to authors or networking with colleagues globally for full-text papers that are not accessible may be necessary. DATA EXTRACTION AND ANALYSES Assessing the risk of bias (RoB) is crucial in this context. Many tools are available to evaluate the RoB or quality, each with advantages and disadvantages. SRs focused on psychological interventions present unique challenges for bias assessment, particularly since blinding personnel and assessors are often impractical in these studies. Some articles may lack detailed information on randomization and other critical aspects of the study design. Authors may differ in scoring the RoB assessment tool, and consensus may take time. The RoB 1 tool allows for some subjective bias assessment, while RoB 2 is more stringent, often resulting in a high-risk classification under such rigid criteria. RoB assessment provides an understanding of the quality of the studies included in the review. They help authors decide whether to include ones with very high RoB in the final analyses. Sensitivity analysis is conducted with and without the ones with higher biases. Accordingly, any variations in the final results are presented in the report. Funnel plots and statistical tests like Egger’s test are utilized to assess the likelihood of publication bias. Thus, the team must include authors proficient in using relevant software to generate these results. Data extraction from studies must be carried out meticulously to avoid errors or misinterpretations. This process is labor-intensive and requires patience. Authors may sometimes face uncertainty about which data to extract, and missing data in some articles can complicate the synthesis of results. In addition, variations in how individual studies present their results pose challenges that require expertise. For example, some studies may report changes in tool scores, while others provide pre- and post-intervention scores for both experimental and control groups. Statistical methods should be chosen in consultation with statisticians experienced in meta-analyses. It is essential to ensure that the studies included in the meta-analysis are sufficient to justify the pooling of results. Sensitivity analyses are crucial for demonstrating the robustness of the findings, requiring careful consideration to reflect the authors’ depth of understanding. Understanding and interpreting heterogeneity is another key aspect of meta-analysis, best done with the assistance of statisticians. Individual studies in an SR often have limitations and methodological differences, contributing to heterogeneity, which must be addressed effectively.[6] Despite the significant efforts invested, there can be disappointment if the research question is not fully answered or the findings are not useful for clinicians. However, the rigor of the methods and processes followed is equally important, even though many reputable journals tend to prioritize the study’s findings. OVERCOMING GENERAL OBSTACLES It is well known that conducting SR involves laborious work. Careful project planning, securing adequate resources, and distributing tasks among motivated team members are crucial for success. Involving experts who can guide the process can make it easier. Personal challenges such as procrastination, limited availability of a committed team, and sudden need to focus on other priorities can become significant barriers. A lack of knowledge regarding the procedure can create more difficulties in the already tedious task. Constantly revisiting tasks can lead to burnout, especially when faced with persistent obstacles. A team with researchers who know how to conduct SRs, an experienced statistician, peer review at every stage, and meticulous note taking at each step can significantly help overcome the above-mentioned challenges. Regular breaks may be required. CONCLUSION A significant proportion of PROSPERO registrations are reported as having an “ongoing” status, indicating that many protocols are not being converted into published systematic reviews.[7] Factors such as a lack of funding and other resources can hinder motivation to proceed. In addition, the lengthy process and varying steps based on the type of work involved can be daunting. To navigate these challenges, quick decision-making and effective note-taking can help alleviate many difficulties. There may be times when it is necessary to pause, convene a meeting, discuss issues, reach a consensus, and then move forward. Throughout this process, it is essential to maintain objectivity to minimize bias. All procedures should be documented to ensure replicability by anyone, anywhere in the world. Rather than viewing obstacles as setbacks, they should be seen as opportunities for growth. The quality of the adopted methodology should take precedence over the results. Instead of fixating solely on outcomes, it is important to enjoy the collaborative journey as a team.