How many sounds is a picture worth? A dual process perspective on the differences between memory for environmental sounds and pictures

Recognition memory on standard old/new tests is typically better for pictures than for sounds. To investigate this modality effect, we examined the contributions of recollection and familiarity processes to recognition memory of auditory and visual objects (environmental sounds and colored line drawings, respectively). To gain some insight into the source of the modality difference, we investigated whether the modality effect was larger for one process than the other. Indeed, in Experiment 1, recollection was greater for visual than auditory objects, but the modality effect on familiarity was much smaller. We replicated this pattern in all subsequent experiments. Focusing on recollection, we examined the consequences of hindering encoding during study with a divided attention task. Surprisingly, Experiments 2 - 4 showed that divided attention at study substantially reduced visual recollection but had little effect on auditory recollection, to the point that visual recollection in the divided attention condition was similar to auditory recollection in the full attention condition. This unusually small effect of divided attention on auditory recollection aligns with prior research showing fundamental differences in recognition of sounds and pictures. Further, it suggests that effects we have taken as regularities in recognition (such as divided attention) may not apply to all modalities.