Examining the Meaning and Methodological Characteristics of the Systematized Review Label: A Scoping Review

Objective – In 2009, a typology by Grant and Booth introduced the concept of a systematized review in which authors (typically students) selectively employ various elements of the systematic review process. As academic librarians who help research teams to select and conduct a variety of evidence synthesis types, we have fielded inquiries from teams interested in conducting systematized reviews for publication. While the typology is widely referenced, we were aware of no formal methodological guide to a systematized review type. The objective of this scoping review is to identify and describe the extent of published systematized reviews, and to 1) identify and collate, where available, sources used for the conceptualization and conduct of the systematized reviews, 2) determine if explanations provided were based on constraints, and 3) describe common methodological characteristics. Methods – Articles titled as a systematized review that attempted the collocation and synthesis of literature and included an adequate description of their methodology were included. We searched on September 1, 2023, the following five sources: Google Scholar, Lens.org, Web of Science Core Collection, Scopus, and MEDLINE. We performed screening and data extraction in duplicate. Data extraction elements included common methodological characteristics relating to various steps of the evidence synthesis process. Descriptive, aggregate statistics, and categorization of reasons for selecting the systematized review type were the primary analysis for this review. Results – Our review found 171 systematized reviews that met inclusion criteria. These were published between 2013 and 2023, and the number has increased each year. Resources or constraints were mentioned in only 32 included reviews, and only 19 had a single author. The methodological attributes of published systematized reviews vary significantly. A small number (15) of reviews searched only one database or source, while the majority searched between 2 and 6 sources. The majority (134) provided no search details or a non-reproducible search strategy. Only 36 included reviews mentioned librarian involvement. Conclusion – Librarians should stay abreast of emerging evidence synthesis practices and the context in which researchers operate. It is unclear why and how authors choose the systematized review type because most did not report a source for their conceptualization, nor justify their choice. The majority of published reviews did not connect their methodological departures to the kinds of resource constraints typically associated with student work. The expectation that the elements of the systematic review process that authors chose to incorporate should be conducted and reported according to standards was not widely met. Librarians, as methodological guides, participants, or evaluators can play a key role in reinforcing the expectations of this review type and the standards that should be met when publishing a systematized review. Publishers and consumers of systematized reviews are advised to consider how each modification of standard systematic review methodology may introduce bias and affect the value of the evidence presented in published systematized reviews.