Dual-Modal 3D Human Pose Estimation using Insole Foot Pressure Sensors

We propose a real-time dual-modal 3D pose estimation system leveraging a single pair of insole sensors. Unlike conventional methods requiring fixed cameras or bulky wearable sensors, our approach offers minimal and natural setup requirements. The system uses pressure and IMU sensors embedded in insoles to capture foot pressure distribution and 6 DoF acceleration, estimating 3D full-body joint positions via a two-stream transformer network. We introduce a novel double-cycle consistency loss and a cross-attention module to learn the relationship between 3D foot positions and their pressure distributions. Our experiments show that the proposed system performs comparably to top approaches using more IMUs and even outperforms some third-person-view camera-based methods in certain scenarios.