DeforHMR: Vision Transformer with Deformable Cross-Attention for 3D Human Mesh Recovery

Human Mesh Recovery (HMR) is an important yet chal-lenging problem with applications across various domains including motion capture, augmented reality, and biome-chanics. Accurately predicting human pose parameters from a single image remains a challenging 3D computer vision task. In this work, we introduce DeforHMR, a novel regression-based monocular HMR framework designed to enhance the prediction of human pose parameters using deformable attention transformers. DeforHMR leverages a novel query-agnostic deformable cross-attention mecha-nism within the transformer decoder to effectively regress the visual features extracted from a frozen pretrained vision transformer (ViT) encoder. The proposed deformable cross-attention mechanism allows the model to attend to rel-evant spatial features more flexibly and in a data-dependent manner. Equipped with a transformer decoder capable of spatially-nuanced attention, DeforHMR achieves state-of-the-art performance for single-frame regression-based methods on the widely used 3D HMR benchmarks 3DPW and RICH. By pushing the boundary on the field of 3D hu-man mesh recovery through deformable attention, we intro-duce an new, effective paradigm for decoding local spatial information from large pretrained vision encoders in computer vision.