Interaction effect of reproducing spatialized human-induced and environmental sounds from nearby sources on the social Simon effect

• The interaction effect of 3D spatialization of ES and HiS on human perception was investigated. • Both SSE and subjective impressions of others’ presence are affected by spatialization of HiS and ES. • A sense of presence can be controlled by ES and HiS. Advancements in virtual reality technology have led to a corresponding increase in demand for virtual environments with multimodal reality in terms of not only visual, but also auditory, tactile, olfactory, and gustatory stimuli. From an auditory standpoint, to reproduce a more realistic space in which audio communication with others occurs, the influence of acoustic reproduction methods on the sense of presence of another person needs to be examined. Assuming a virtual environment in which either artificial sounds generated by others or nonhuman background sounds generated in the surroundings, or both, coexist, the present study investigated the behavioral and psychological effects of the three-dimensional spatialization of human- and nonhuman-induced background sounds on users. First, human- and nonhuman-induced background sounds were reproduced in three different ways: binaural, quasi-binaural, and diotic playback. Next, experiments were conducted to evaluate the subjective impression of the sense of someone being present in the environment and to confirm the social Simon effect (SSE), which is excited by the imagined presence of others nearby. The results revealed that both the behavioral emergence of SSE and subjective impressions of the presence of others coexisting nearby are strongly influenced by the relative degree of spatialization of both human- and nonhuman-induced environmental sounds.