How does subjective confidence influence multisensory integration?

Research on multisensory perception has uncovered the fundamental computations underlying the integration of information from diverse sensory sources. Prevailing theory posits that information from diverse sensory modalities undergo a weighting process, wherein the contribution of each sensory input is determined by the uncertainty associated with its perceptual representation in the brain. Take audio-visual localization, for instance, where visual cues typically yield lower uncertainty estimates than auditory cues, visual cues are assigned greater weight than auditory cues in the final audiovisual percept. However, very little is known about the relationship between metacognition and multisensory integration. In the current study, we are investigating how subjective uncertainty (measured as a confidence judgment ) in the visual modality affects how a visual stimulus is weighted in audio-visual integration. In Experiment 1, observers reported the mean position of a cloud of dos on each trial. To isolate subjective confidence, we aimed to manipulate participants’ confidence levels while keeping their performance the same by changing the spatial distributions of the dots (i.e., variability) and the distance between the dot cloud’s epicenter and central fixation (i.e., offset). Participants (N = 18) reported their location judgment (left/right) and their confidence (1-4) in the location decision using a single button press. We observed higher confidence ratings with larger dot variability even though there was no significant change in discrimination accuracy due to compensatory changes in dot offset. Thus, we are able to induce a dissociation between subjective confidence and objective performance in a visual localization task. A secondary experiment is underway to test the influence of this subjective confidence bias on audio-visual weighting in a paradigm using concurrent auditory cues. In sum, we used a novel manipulation to demonstrate a confidence-accuracy dissociation in visual localization, a critical first step to understanding how subjective confidence influences multisensory integration.