Semi-Supervised Bird’s-Eye-View Mapping for Self-Balancing Exoskeletons Using RGB-D Sensing

We present a novel mapping approach for lower-limb exoskeletons that generates real-time, robot-centric bird’s-eye view (BEV) occupancy maps to support safe and efficient local navigation. This work focuses on a self-balancing wearable humanoid exoskeleton, where BEV mapping is essential for enabling autonomous balance control, footstep planning, and adaptive navigation in complex, real-world environments. The proposed method explicitly incorporates camera motion alongside RGB-D observations to improve mapping accuracy under the dynamic conditions introduced by leg-mounted sensors. To meet the computational constraints of embedded platforms, the model is optimized for real-time operation and can effectively track dynamic elements such as moving pedestrians. We further introduce a semi-supervised framework that combines simulation-based supervised training with unsupervised learning on real-world data, enabling robust generalization despite limited ground-truth labels. Experiments in both simulated and real environments confirm that the model achieves fast inference (17 ms) with low memory consumption (600 MB). Moreover, the model remains robust to the exoskeleton’s motion as well as various sources of environmental noise.