Can Sounds Change the Perception of Pictures? Exploring the Influence of Semantic and Spatial Congruency on Aesthetic Perception

Aesthetic perception involves inputs from various senses, and the aesthetic perception of one sensory modality can be affected by information from another sensory modality. This phenomenon can be explained by crossmodal associations, with semantic and spatial correspondence playing important roles. However, the effects of both semantic and spatial congruency on aesthetic perception remained unclear. Here, a pre-registered experiment was conducted to investigate the influences of semantically and spatially congruent sounds on aesthetic perception of the pictures presented simultaneously and whether these influences were mediated by processing fluency. Participants were asked to evaluate the liking, aesthetic value, and processing fluency of pictures presented simultaneously with sounds under four conditions (spatial congruency/incongruency and semantic congruency/incongruency). The results indicated that the semantically congruent sounds increased the processing fluency, thereby leading to higher liking and aesthetic evaluation of pictures. However, a significant effect of spatial congruency was not observed. The null finding may suggest that the effect of spatial congruency was likely too small to be practically meaningful in aesthetic perception with unlimited response time, which might have caused participants to respond after the influence of spatial congruency had gradually faded. Another possible explanation is that, in comparison to spatial cues, the manipulation of semantic congruency better aligns with individuals’ learned expectations. Therefore, semantic cues that are more in line with expectations may be more easily observed to influence subjective evaluations.