DCFormer: Divide-and-Conquer in 3D Human Pose Estimation Tasks

Monocular 3D human pose estimation based on Transformer has recently attracted significant attention. Intuitively, general 3D human pose estimation simultaneously learns the camera coordinates in three dimensions. However, this approach highly couples two different tasks: (1) the scale estimation from 2D pixel to 2D camera coordinates in xy-axes; and (2) the depth estimation from 2D pixel coordinates to relative root depth in the z-axis. This results in the inability to adjust flexibly for different tasks, let alone achieve the best for each task. To this end, we propose a "Divide and Conquer" model framework, which decomposes the relevant learning tasks into the scale and depth estimation task. This way, we can train the two tasks separately to optimize both. The decoupled structure supports improving modules for a specific task to get a boost, such as adding masks to the scale learning task to enhance generalization and prevent overfitting, but without considering the negative impact on another. To verify the effectiveness of the proposed scheme, we conducted extensive experiments on two popular benchmark datasets, Human3.6M and MPI-INF-3DHP, achieving SOTA and reducing the number of parameters by 37.5%. The code is available at https://github.com/Mitanies/DCFormer.