Comparison of Psychological Data between Populations, A Measurement Perspective

In cross-cultural psychology there is a strong tradition of psychometric analysis of bias and invariance (equivalence) after collection of data in distinct populations with identical instruments developed in one population. This implies a focus on measurement outcomes and an operationalist orientation to science. Here we explore how a focus on joint development and content validity of instruments combined with relaxation of the demand of item identity can provide an additional perspective. Three points are addressed:(1) Any comparison between participants from two, or more, populations presumes a common quality (property, attribute) and requires a common measurement scale, a “standard of comparison”.(2) Stevens (1946) four levels of measurement are distinguished for a standard of comparison; this level (nominal, ordinal, interval, ratio) determines to which kinds of psychometric comparisons data are open.(3) A standard of comparison can be based on an existing instrument that is transferred from one population to some other population, with or without modification. It can also be a newly developed instrument that may be based, but (we argue) need not be based, on a set of stimuli or items that is identical across populations.Three considerations are mentioned: (i) representation of all participating populations at all stages of instrument development, (ii) emphasis on content validity, and (iii) various tools such as item banks and item parameters from IRT models that can help to derive a comparison standard from only partly overlapping item sets.