Methodological rigor of rare disease guidelines in Europe: a scoping review and critical appraisal

Abstract Purpose This study aims to evaluate methodological rigor of clinical practice guidelines developed, endorsed or displayed by the European Reference Networks (ERNs) across a broad spectrum of rare diseases. Given the well-recognized challenges of guideline development in rare conditions—including limited high-quality evidence, difficulties in formulating clinical questions, and barriers to meaningful patient involvement—we assess how these issues manifest in ERN guidelines to identify best practices. Specifically, the study seeks to generate insight into methodological approaches related to framing clinical questions, managing evidence scarcity, evaluating the strengths and limitations of available evidence, and incorporating patient perspectives, while highlighting shared methodological challenges across diverse clinical domains. Methods We systematically searched all 24 ERN websites (June 2024) for English-language rare disease guidelines published since 2010. Eligible documents required full text, methodological description, and a disease prevalence ≤ 1:2000. Two reviewers independently assessed methodological quality using the AGREE II “Rigor of Development” domain, with scaled scores calculated per guideline. Guidelines scoring ≥ 70% formed an exemplary sample of guidelines with sufficient methodological rigor, which was qualitatively analyzed for approaches to question formulation, evidence scarcity, evidence appraisal, and patient involvement. Results From 205 retrieved documents, 149 guidelines met eligibility criteria. Methodological rigor was generally low, with a mean AGREE II scaled domain score for methodological rigor of 36.2%. Only nine guidelines scored ≥ 70% and were included in the exemplary guidelines sample. Highest-scoring items concerned consideration of benefits and harms, while updating procedures and external review scored lowest. Guidelines in the exemplary sample demonstrated clearer question formulation, broader evidence inclusion strategies, transparent evidence appraisal, and varied but present approaches to patient involvement. Conclusions This review shows that methodological rigor in rare disease guidelines remains suboptimal. Poor reporting of updating procedures and external review highlight clear opportunities for easy improvement. Strengthening external peer review and adopting strategies seen in high-scoring guidelines—such as clear question formulation, broader evidence inclusion, transparent appraisal, and structured consensus methods—may enhance methodological rigor. Knowledge gaps remain in framing questions, search strategies, and patient or caregiver involvement, underscoring the need for tailored methodological guidance for rare disease guideline development.