More Than a Coin Flip: Forced-Choice Performance Above the Binomial Mean and Word-Frequency Bias on the Word Memory Test Under Sham Random Responding

Abstract Below-chance performance on forced-choice performance validity tests (PVTs) is typically interpreted against a binomial model that assumes a .50 probability under uninformed responding. We tested whether this assumption holds for the Word Memory Test (WMT) and whether item-level lexical properties account for departures from the binomial expectation using a sham extrasensory perception (ESP) paradigm to elicit uninformed (i.e., “random”) responding. Fifty-five undergraduates completed the WMT Immediate Recognition (IR) trial without prior exposure to the study list. Total scores and item-level responses were tested against exact binomial expectations. Pair-level lexical contrasts (target minus foil) for log word frequency, age of acquisition, concreteness, and orthographic distance were derived from the English Lexicon Project and examined as predictors of item selection in ordinary least squares (OLS) and generalized estimating equation (GEE) models. Mean IR performance was 24.15 of 40 (60.4% correct), exceeding the binomial mean, t(54) = 8.40, p < .001, d = 1.13. Eighteen IR items deviated from .50 after false-discovery-rate correction, predominantly toward the keyed-correct word. Target-foil word-frequency asymmetry was the dominant predictor of IR item selection (bivariate r = .56; OLS and GEE p < .001); concreteness and orthographic distance were unrelated to IR selection, and age of acquisition did not predict IR selection after frequency was controlled. Under attempted uninformed responding, WMT IR performance exceeded the nominal 50% binomial expectation by 1.31 binomial standard deviations (a magnitude exceeding the minimum threshold for practical significance). Item-level deviations were associated primarily with target-foil word-frequency differences. The findings identify a lexical feature relevant to test construction and forensic interpretation.