EvTransPose: Towards Robust Human Pose Estimation via Event Camera

To overcome the interference caused by varying lighting conditions in human pose estimation (HPE), significant advancements have been made in event-based approaches. However, since event cameras are only sensitive to illumination changes, static human bodies often lead to motion ambiguity, making it challenging for existing methods to handle such cases effectively. Therefore, we propose EvTransPose, a novel framework that combines an hourglass module for global dependencies and a pyramid encoding module for local features. Specifically, a transformer for event-based HPE is adopted to capture the spatial relationships between human body parts. To emphasize the impact of high resolution on HPE tasks, this work designs the cascading hourglass architecture to compress and recover the resolution of feature maps frequently. On this basis, an intermediate-supervision constraint is incorporated to guide the network in aggregating sufficient features during the intermediate stages, which ensures better feature refinement and enhances overall performance. Furthermore, to facilitate a thorough evaluation of our method, we construct the first event-based HPE dataset with RGB reference images under diverse lighting conditions. Comprehensive experiments demonstrate that our proposed EvTransPose framework outperforms previous methods in multiple aspects.