A six-phase AI-expert framework for evaluating policy coherence in sustainable tourism

This study introduces a six-phase AI–expert evaluation framework, grounded in computational interpretivism, to assess the coherence between national and international sustainable tourism policies, using the Croatian Strategy for Sustainable Tourism as a case study. Leveraging Anthropic Claude 3.5 Sonnet, a state-of-the-art generative AI model, the framework integrates natural language processing, vector-based similarity search, and expert validation to extract key policy areas, formulate evaluation questions, and assess alignment with the European Union’s principal tourism policy documents: the Transition Pathway for Tourism, European Agenda for Tourism 2030, and EU Strategy for Sustainable Tourism. The analysis produced a validated set of 68 binary and 27 qualitative evaluation questions, developed through iterative AI–expert collaboration and verified through quantitative similarity measures (Jaccard coefficients of 0.714 for key areas and 0.904 for topics). Three domain experts independently validated all phases, achieving substantial inter-rater reliability (Fleiss’ κ = 0.717). Results confirm that AI can substantially enhance efficiency, scalability, and transparency in policy analysis, while expert oversight remains indispensable for contextual interpretation, ethical validation, and policy relevance. The study advances theoretical understanding of hybrid human–AI epistemologies and contributes a replicable, auditable model for policy coherence evaluation in complex governance systems. Beyond tourism, the framework demonstrates how AI-assisted analysis can strengthen evidence-based policymaking, supporting adaptive governance and cross-level policy alignment in sustainability-driven sectors.