Experimental Researchers Do Not Know it, but Item Reliabilities are Crucial for Prediction

The importance of test reliabilities for predicting criterion variables has been well-established by psychometricians and is familiar to experimental researchers too. Thus, the standard operating procedure for experimental researchers includes assessing and reporting Cronbach’s alpha. However, experimental researchers generally ignore item reliabilities. And yet, item reliabilities can be argued as much more important than indices of single-administration whole test reliabilities, such as Cronbach’s alpha. The present goal is to make that argument and detail the complications that arise upon considering item reliabilities. Item reliabilities interact complexly with true itemcriterion correlation coefficients, true interitem correlation coefficients, the number of items, and whether the researcher engages in amalgamating or unamalgamating test items. The standard operating procedure should include the assessment and reporting of item reliabilities