Analyzing gaze and hand movement patterns in leader-follower interactions during a time-continuous cooperative manipulation task

In daily life, people often interact by taking on leader and follower roles. Unlike laboratory experiments, these interactions unfold naturally and continuously. Although it is well established that gaze typically precedes object manipulation, much less is known about how gaze-hand patterns evolve in interactive settings where one person must take the other's actions into account. Here, we examine predictive, planning-related behavior in a two-player tabletop game called "do-undo." Participants alternated as Leader and Follower. The Leader performed simple pick-and-place actions to alter the arrangement of objects, while the Follower used other objects to restore the previous configuration. We recorded eye and hand movements, along with object trajectories, using a system that combined eye tracking with multi-camera motion capture. Touch sensors on the players' hands provided precise timing of contacts, allowing us to segment cooperative action into well-defined temporal intervals. As expected, eye fixations consistently preceded manipulation, but clear role differences emerged. Leaders looked more often and earlier at target objects. Further, Leaders' gaze anticipated not only their own actions but also those of the Followers. Leaders also more frequently checked the outcome of the do-undo sequence. Both roles showed gaze patterns consistent with memorization, but alternating gazes between objects and destinations were much more common in Leaders. Some patterns suggested longer-term planning beyond the immediate action. These findings reveal distinct decision-making and planning strategies in Leaders and Followers. Leader-centric interactions, highlighted by Leaders considering not only their own next moves but also their partners' potential actions, shed light on the complex cognitive processes that underlie everyday human interaction.