Applications of the ErrVarNorm index in case 1 best-worst scaling and data quality insights

The present research extended Llobell et al. (2025) [Fd. Qual. Pref., 10533] who introduced normalised error variance (ErrVarNorm; EVN) as an index of participant response consistency in Case 1 Best-Worst Scaling (BWS). As EVN increases through its 0 to 1 range, response consistency and, in turn, the BWS data quality increase. This easy-to-calculate index was applied to online data collected from 4587 adults from the United Kingdom. The key findings included: 1) Researchers can establish a tolerance for post hoc exclusions (%) and calculate a corresponding EVN cut-off value; 2) Men aged 18–19 years old were most at risk of exclusion based on lower EVN values; 3) Excluding participants with lower EVN value changed BWS object scores and study conclusions; 4) EVN values were not strongly correlated with self-reported BWS task perceptions; and 5) EVN can be used as a general indicator of individual-level survey data quality. EVN = 0.5 was used as the value for participant exclusions, but the results showed that this cut-off may need to be study-specific. Beyond further characterising the EVN index, the present research contributes to the discourse on data quality in online consumer research and advocates for the perspective that data quality exists on a continuum and that researchers are the ones who are primarily responsible for defining needed data quality levels in a focal study and then transparently disclosing steps taken to achieve this. • Applies the ErrVarNorm (EVN) index developed for Case 1 Best-Worst Scaling (BWS) studies. • EVN (0 to 1) provides a measure of respondent consistency and individual-level data quality. • EVN cut-off for post hoc participant exclusion should consider study-specific factors. • Higher EVN values (greater response consistency) were less common in younger men.