Enhancing transparency in high‐stakes English language assessment: A mixed‐methods synthesis of empirical evidence and stakeholder perspectives

Abstract Transparency in high‐stakes English language assessment has become crucial for ensuring fairness and maintaining assessment validity in language testing. However, our understanding of how transparency is conceptualised and implemented remains fragmented, particularly in relation to stakeholder experiences and technological innovations. This study addresses this gap through a mixed‐methods synthesis of empirical evidence and stakeholder perspectives, combining systematic review methodology with primary data from a pilot study of a computer‐adaptive English language test. The research employed a comprehensive approach following PRISMA guidelines, integrating findings from systematic database searches (2000–2024) with empirical evidence from test takers ( n = 299) and test centre managers ( n = 41). Factor analysis of test‐taker responses revealed six key dimensions of assessment transparency: task confidence, test‐taking confidence, time management, perceived difficulty, instructional clarity, and task familiarity. Cluster analysis identified four distinct stakeholder response patterns, with significant variations across demographic groups. The synthesis of systematic review and empirical data analysis reveals that transparency has evolved from a technical consideration into a multi‐dimensional construct that fundamentally shapes stakeholder experiences. Key findings demonstrate that different stakeholder groups engage with transparency initiatives in systematically different ways, that technological transparency has emerged as a fundamental dimension of stakeholder experience, and that transparency requirements evolve in response to external pressures. The COVID‐19 pandemic particularly highlighted how crisis conditions can rapidly transform transparency needs. These findings have significant implications for assessment design and implementation. They suggest that assessment providers must develop ‘layered transparency’ approaches that accommodate differentiated stakeholder needs while maintaining consistent standards. The research also indicates that future assessment frameworks, especially those informed by AI, must balance competing demands while ensuring all stakeholders have clear understanding of and engagement with high‐stakes assessment processes that profoundly affect their lives. Context and implications Rationale for this study Our understanding of transparency in high‐stakes English language assessment is fragmented. This study synthesises evidence and stakeholder views to build a more coherent model. Why the new findings matter This research reveals assessment transparency as an evolving, multi‐dimensional construct, demonstrating varied stakeholder engagement and highlighting the impact of technology and crises on transparency needs. Implications for researchers, policy makers and practitioners The findings advocate for layered transparency approaches which are responsive to diverse stakeholder needs by test developers and providers, with specific recommendations addressing time management, technical usability, communication clarity, cultural/linguistic accessibility, test security and fairness, and better preparation resources. For policymakers, the study underscores the need for assessment frameworks supporting adaptable and equitable transparency standards. For researchers, it identifies crucial areas for future inquiry, including the impact of transparency on test validity/security, and navigating emerging technologies like AI within transparent frameworks.