Assessing transparency and methodological precision in variable measurement within organizational research: implications for validity

Abstract This study addresses the call for enhanced transparency in methodological reporting by critically assessing methodological rigor and transparency, particularly in variable measurement, within primary cross-sectional survey-based research in organizational studies. Through a combined approach of scoping review for systematic literature identification and qualitative analysis, instances of inadequate reporting transparency and lack of methodological rigor are revealed. Analyzing data from 68 published studies, the findings highlight unclear construct definitions and a consistent absence of theoretical limitations when presenting study findings, pointing to a need for better reporting transparency. Additionally, the study uncovers the presence of elusive links between variable constructs and measures, and misplaced claims of prior validation for measurement instruments, indicating a lack of methodological rigor. Collectively, these findings challenge the interpretation and validity of research results, potentially leading to misconceptions and misinterpretations about what is known about the variable(s) in question. In turn, this may lead to researchers and practitioners basing their research or practice on flawed assumptions about what a group of studies is saying or indicating. While acknowledging prior recommendations on variable measurement and transparency, this study brings renewed focus to these areas. It also explores the role of measurement proficiency, debating whether the observed limitations reflect ethical concerns or are indicative of researchers’ capabilities in variable measurement, validity assessment, and reporting.