R-Metaverse: multimodal social cues for real-world-centered telepresence–MR collaboration with telepresence robots

Remote collaboration in live physical workspaces requires direct access to the evolving scene and shared task information anchored to it. Telepresence provides embodied access but lacks a shared spatial interaction layer, whereas fully virtual collaboration requires reconstructing the workspace. R-Metaverse (Ritsumei Real-World Robotic Metaverse) is a telepresence–MR system design centered on the physical workspace. A remote participant operates a telepresence robot and views its camera image augmented with virtual objects through a head-mounted display (HMD), while an on-site participant views the same spatially registered virtual objects through a smartphone. The robot–mobile configuration creates viewpoint and embodiment asymmetries that can reduce social presence. Robot–robot collaboration makes partner distance and approach difficult to perceive because of robot-camera field-of-view limitations. To restore missing cues, we implement facial-expression display, motion-synchronized avatar overlay, spatially localized audio, and distance-linked vibrotactile feedback. The first three target robot–mobile collaboration; the fourth targets robot–robot collaboration. Ablation experiments show that facial-expression display improves affect understanding and closeness, while distance-linked vibrotactile feedback supports co-presence and spatial awareness under field-of-view constraints. Avatar, audio, and vibration-pattern effects are interpreted as exploratory after Holm correction. These results suggest that gap-specific multimodal cues can partially compensate for reduced social presence in real-world-centered telepresence–MR collaboration.