Comparing apples to oranges to bananas: A big data approach to understanding the joint influences of stimulus properties, trial history, and individual differences

Research into the mechanisms of visual search has often explored how trial-level differences in the stimuli (e.g., set size, target salience) affect performance. However, performance is also influenced by trial history effects (e.g., priming, hysteresis) and individual differences (e.g., variation in task capacity). Each of these three factors (stimuli, history, individual differences) has been examined independently, but understanding their comparative importance and how they may interact is highly informative in a wide range of applied scenarios, for example in making resource allocation decisions between user-interface design, task training, or personnel selection, respectively. Unfortunately, it has traditionally been extremely difficult to evaluate the comparative contribution to task performance of each factor simultaneously, as this requires a large dataset with sufficient variation in trial-by-trial stimuli attributes, exposure to task conditions, and participant characteristics. The current study leveraged a massive dataset of visual search performance (~3.8 billion trials, ~15.5 million individuals) from a mobile game version of an airport security visual search task (Airport Scanner, Kedlin Co.) to quantify and compare the variance in performance accounted for by three factors of performance: 1) trial-by-trial stimuli search array features (e.g., current trial target identity and array set size), 2) trial history specific to each individual’s experience of the task (e.g., cumulative exposure to target), and 3) individual differences in task performance aptitude (e.g., participant-specific target hit rate). Each of these three factors was found to strongly contribute to performance, but, importantly, the nature and magnitude of their influences varied. In particular, individual differences were found to be an extremely large, and relatively stronger, predictor of performance variance. Precise quantifications of each factor’s comparative contribution across several task contexts are provided.