STAR Recommendations: A novel framework for generating recommendations

Clinical practice guidelines (hereinafter referred to as “guidelines”) refer to evidence-based recommendations for delivering optimal medical services to patients, considering the benefits and disadvantages of various interventions.[1] In recent years, there has been a significant surge in the production of guidelines globally: the number of guidelines indexed in PubMed has increased nearly 50-fold between 1990 and 2023.[2] The number of guidelines led by Chinese contributors has also increased almost 40-fold during the past two decades.[3–6] Considering this escalating global trend in guideline production, it is even more crucial to address the concerns about their quality. High-quality guidelines enhance the scientific basis of clinical decision-making and play a pivotal role in standardizing diagnostic and treatment practices among healthcare professionals. Conversely, poorly standardized or incomplete guidelines, and guidelines affected by substantial conflicts of interest, can misguide clinical decision-making and squander valuable medical resources. Recent surveys indicate that both methodological and reporting quality of guidelines are generally subpar,[7] which limits their adoption and implementation in clinical practice.[8–10] Guidelines can be broadly categorized into three groups according to their formulation cycle and development process: standard guidelines, rapid advice guidelines, and rapid recommendations (such as BMJ Rapid Recommendations). Standard guidelines have a longer formulation and revision period with extensive content and complex production steps, typically taking 6 months to 2 years.[11] Rapid advice guidelines[12] and BMJ Rapid Recommendations[13] are developed in shorter cycles, usually within 1–3 months. While traditional guidelines are rooted in systematic evidence synthesis to support decision-making in stable clinical environments, rapid advice guidelines and rapid recommendations offer the advantage of accelerated development processes to address urgent healthcare needs. Despite their distinct development frameworks and operational timelines, all three types of guidelines face common challenges. For instance, the majority of guidelines are published without undergoing a thorough quality assessment, which may result in the omission of certain critical and standardized steps during the development process. In addition, the implementability of the recommendations developed according to the existing methods is not assessed before publication, and no pilot implementation is conducted. Consequently, the readers and users cannot assess the quality or applicability of these guidelines well. Moreover, artificial intelligence (AI) technology has great potential to enhance the efficiency of developing guidelines; however, its application within existing guideline development processes has yet to be sufficiently explored and validated. To promote the development and dissemination of high-quality guidelines, this paper proposes a novel methodological framework known as the STAR Recommendations, grounded in the principles of scientific, transparency, and applicability. Drawing on the STAR (Scientific, Transparent, and Applicable Ranking) tool launched in China in 2022, this approach integrates the tool as a pivotal early-assessment component, thereby providing more timely and effective recommendations in clinical practice.[14] The aim of this article is to present how rapid recommendations can be formulated in a standardized way based on the principles of STAR. The involvement of AI in the development of STAR Recommendations is encouraged, while transparent reporting on the details of any AI utilization and engagement is mandatory. Besides, the characteristics and formulation process of STAR Recommendations, as well as the potential limitations and precautions needed in their use, are introduced. Features of the STAR Recommendations Management and coordination: The development of the recommendations is managed by the STAR Executive Committee, coordinated by the STAR Secretariat, and led by the respective STAR Specialty Committee. Record number: Each recommendation is assigned a unique record number. Timeline: The development of the recommendations typically requires 1–3 months, and they should be updated every 3–6 months. Sources of recommendations: Recommendations are derived from relevant guidelines, systematic reviews, original research, and expert opinions.[15] Consensus method: Consensus meetings to formulate the final set of recommendations can be conducted online or in person. Implementation testing: The recommendations should be tested in 1–3 hospitals to collect feedback before submission. Comprehensive evaluation: An evaluation report according to the STAR principles must be submitted prior to submission of the recommendations.[3,4] Release process: STAR Recommendations will be jointly published by members of the relevant STAR Specialty Committee along with medical and health societies, associations and medical institutions. AI integration: AI can be incorporated at all stages while maintaining transparent reporting on its level of involvement. Multilingual and multi-version support: Disseminating the recommendations in multiple languages is encouraged. In addition to the professional version (for healthcare staff), a patient version or other versions of the recommendations may also be developed. Formulation process of STAR Recommendations Figure 1 illustrates the technical pathways in the formulation process of STAR Recommendations. Details of the different steps of the formulation process are given in the next sections.Figure 1: Flowchart for the development of STAR Recommendations. AI: Artificial intelligence; STAR: Scientific, Transparent, and Applicable Ranking.Proposing topics and submitting initial materials STAR specialty committees may initiate STAR Recommendations either independently, or jointly with medical associations or institutions. Prior to commencing the development, three documents should be completed: (1) a concise statement of the clinical problem emphasizing its significance and priority (a research report on the clinical problem can be attached); (2) a comparative analysis of current guidelines as well as consensus recommendations related to this specific clinical issue; and (3) a comprehensive plan with a schedule for consideration. Review conducted by the STAR secretariat and executive committee The specialty committee that initiated the STAR Recommendations shall submit the materials to the STAR Secretariat for preliminary review, primarily focusing on formal aspects. This review process typically takes 1–3 working days. Upon successful completion of this stage, the secretariat will transfer the materials to the STAR Executive Committee for a comprehensive review within seven working days. The focus of this review is to assess topic selection and implementation plans. Once approved, a record number will be assigned, and the formulation activities will commence. In case of non-approval, topics may be resubmitted after revision, and materials can be supplemented or modified before undergoing further evaluation. Utilization of AI in facilitating information retrieval, assessing evidence, and generating preliminary recommendations AI can be utilized in the following tasks: formulating search strategies for the guidance group; conducting literature search and screening; evaluating the quality of the included literature; summarizing the literature; generating an evidence summary table; and, based on the evidence summary, forming opinions for recommendations. The level of AI involvement depends on the requirements and realities of the specialty area and the development team, but the participation process and key elements should always be fully reported to ensure transparency. Consensus meeting The development group will conduct 1–3 consensus meetings, either online or in person, to discuss the recommendations. The recommendations will be accepted when a certain pre-defined proportion (e.g., 80%) of the members reach an agreement on their content. Testing the implementation in practice After reaching consensus, the recommendations should be implemented on a trial basis in 1–3 representative medical institutions to gather feedback regarding their clinical applicability and implementation. Subsequently, modifications and improvements can be made to enhance the recommendations if necessary. Furthermore, it is essential to explore specific strategies for implementing the recommendations in a larger scale. Comprehensive evaluation The content is assigned a rating according to the STAR principles by the developers to ensure the scientific, transparency, and applicability of the formulation process before submission. The STAR tool encompasses 11 fields with a total of 39 items. STAR Recommendations will not undergo the official STAR evaluation which is conducted every year to appraise and rank all guidelines published in the previous year. Further information on the STAR rating system can be found in previously published articles[14] and the official website of STAR (https://www.star-guidelines.cn/). Submitting, publishing, and disseminating the recommendations The publication in multiple languages is strongly encouraged; and it is advisable to provide different versions tailored for various target users, including healthcare professionals and patients. Whenever possible, the complete text should be published through an open access platform or made otherwise freely available. Frequency and methodology of updates The developers responsible for developing the STAR Recommendations should conduct a comprehensive AI-assisted evidence search every 3–6 months. If any new evidence emerges that could potentially alter the recommendations, it must be thoroughly evaluated and, if necessary, a request for an update must promptly be submitted to both the STAR Secretariat and Executive Committee for review. Once the request is approved, the Specialist Committee can start updating the recommendations according to the production process of STAR Recommendations, and the corresponding section on the STAR website will be subsequently updated. The development process of STAR Recommendations enables faster, more efficient, and transparent generation of recommendations compared to standard guideline formulation process. Compared with existing rapid recommendation development methods, STAR Recommendations has embedded the use of AI more strongly into the process. We believe that the use of AI assistance under rigorous guidance and following strict and transparent reporting criteria can substantially accelerate the development of the recommendations. Furthermore, the methodology of developing STAR Recommendations is not only applicable in clinical settings but also extends to public health, health policy, and social sciences. This new framework still also faces some limitations. For instance, the STAR tool is presently used only by experts from the Chinese STAR Specialty Committee to evaluate guidelines published by Chinese authors, and the newly developed STAR Recommendations method will initially be implemented by Chinese experts. Nonetheless, we are actively establishing an international STAR Specialty Committee and anticipate that STAR Recommendations will be adopted worldwide in the near future. We also plan to develop and issue more detailed guidance for the selection, prioritization, and formulation of clinical questions. We also need to provide a more thorough examination of potential challenges in the implementation of STAR Recommendations, particularly regarding the integration of AI and ensuring transparency in reporting. Finally, as AI methodology is expected to continue to advance rapidly with an increasing availability of different languages, tools and functions, STAR Recommendations will continue to explore new avenues for accelerating the development of high-quality recommendations. Funding This study was funded by the Key program of National Clinical Research Center for Child Health and Disorders in Children’s Hospital of Chongqing Medical University, Chongqing, China (No. NCRCCHD-2023-KP-01). Conflicts of interest None.