Children Sustain Their Attention on Spatial Scenes When Planning to Describe Spatial Relations Multimodally in Speech and Gesture

How do children allocate visual attention to scenes as they prepare to describe them multimodally in speech and co-speech gesture? In an eye-tracking study, Turkish-speaking 8-year-old children viewed four-picture displays depicting the same two objects in different spatial relations as they prepared to describe target pictures depicting left-right relations. Children's visual attention was sustained on the target picture when they were planning descriptions that expressed the spatial relation multimodally in speech and gesture, but not unimodally in speech only. This pattern persisted regardless of the semantic relation between speech and gesture (i.e., for both complementary gestures that disambiguated speech and redundant gestures that supplemented already unambiguous speech). Importantly, visual attention patterns did not differ across description types while children were previewing the displays before message preparation. These results indicate that multimodal message preparation might place different demands on visual attention than unimodal message preparation, possibly due to the affordances of gestures for expressing spatial relations. SUMMARY: We test the idea that the way children attend to spatial relations prior to communication is related to how they end up describing them. Children's attention was sustained on spatial scenes when they were planning multimodal spatial descriptions in speech and gesture, but not unimodal descriptions in speech only. Planning multimodal descriptions was associated with sustained attention on the spatial scenes regardless of the semantic relation between speech and gesture.