Prior visual experience increases children's use of effective haptic exploration strategies in audio-tactile sound-shape correspondences

Sound-shape correspondence refers to the preferential mapping of information across the senses, such as associating a nonsense word like “bouba” with rounded abstract shapes, and “kiki” with spiky abstract shapes (e.g., Ramachandran & Hubbard, 2001, Spence 2011). Such associations are found between auditory and visual stimuli (AV), and between auditory and tactile stimuli, which are touched but not seen (AT). Previous research highlights how atypical visual experience in the blind (e.g. Fryer et al., 2014; Sourav et al., 2019) and naive visual experience in children (Chow et al., 2021) can yield weak or absent AT associations. Such findings imply that visual experience influences AT association, yet, the mechanisms underlying such influence remain unclear. Here we investigate one potential mechanism: seeing the abstract shapes improves haptic exploration by increasing the use of effective haptic strategies and/or decreasing the use of ineffective haptic strategies in children. We examined haptic exploration from videos of a previous study (Chow et.al., 2021). Thirty 6-8 year-old children completed 16 AV and 16 AT trials, with order counterbalanced across participants. Children picked which of two, side-by-side, visual shapes (AV trials) displayed on a screen, or tactile shapes (AT trials) presented hidden from view inside a box, matched a nonsense sound. We quantified the proportion of duration of each of 5 haptic exploration strategies (effective: contour following, clawing, pinching, and poking, and ineffective: sweeping) for each participant, as they explored tactile shapes during AT trials. We found that with prior visual experience (AV trials first), poking, an effective strategy, was dominant whereas without prior visual experience (AT trials first), uncategorizable, ineffective, strategies were dominant. These findings suggest that prior visual experience of abstract shapes in 6-8 year-olds can increase the of haptic exploration, potentially explaining why prior visual experience can strengthen audio-tactile sound-shape correspondences early in development.