The role of generative AI in navigating trade-offs in policy research design: balancing validity, rigour, and innovation

This study systematically examines how generative AI can assist in policy research design by analysing methodological trade-offs. Using Claude Pro, we evaluated research designs in 164 articles from leading policy journals across six dimensions: internal, external, construct, and predictive validity, methodological rigour, and conceptual novelty. The analysis demonstrates that AI can effectively identify methodological trade-offs and suggest design improvements within resource constraints, though its utility varies by research type. The findings advance our understanding of AI’s role as a complementary tool in research design while highlighting the critical importance of human expertise in methodological decisions.