EgoTokens: Egocentric 3D human reconstruction and language-guided shape editing via quantized position tokens in vision-language models

3D human body reconstruction from egocentric images is a challenging task because of environmental and self-occlusion. Downward viewpoints and scale ambiguity can also hinder precise 3D estimation. Current methods mainly rely on specialized regression networks to reconstruct 3D meshes that often lack structural guidance from high-dimensional SMPL-X parameters. In the proposed work, we introduce EgoTokens, a framework that estimate egocentric SMPL-X parameters as a discrete sequence-generation task. Our end-to-end system employs vision-language models (VLMs) to predict SMPL-X body parameters directly from a single image. Each of the 122 SMPL-X parameters is quantized into 1000 bins and represented as an extended vocabulary for training the VLM. To reflect the extended vocabulary in the network, the corresponding embedding and output-projection layers are redefined and trained with multiple learning rates to accelerate convergence of new quantized tokens. The system is trained with a distance-aware objective that intuitively combines the original cross-entropy with an L2 regression loss to capture actual numerical closeness between predicted bins. Standard VLMs with cross-entropy lack the capability to capture this numerical closeness, and they treat close numbers the same as far numbers. A language-instructed semantic shape editing pipeline is also constructed, extending the EgoBody dataset from 150,000 (visible) to 1.3 million instruction-image-parameter samples. This language-instructed reconstruction allows editing without extra region segmentation. On the EgoBody dataset, the proposed system achieves a PA-MPJPE of 29.49 mm and PA-MPVPE of 28.20 mm for fine-tuning InternVL2-8B, and a PA-MPJPE of 33.43 mm for Qwen2.5-VL-7B. The results show that the proposed system outperforms the specialized regression and transformer-based baselines for both InternVL2-8B and Qwen2.5-VL-7B. Zero-shot evaluation on the EHF dataset achieved a PA-MPVPE of 33.85 mm, showing the trained model’s generalization from egocentric to third-person view with complex poses. The results show that tuned VLMs with distance awareness is a promising direction for 3D human reconstruction and shape control within a single framework. Code, Model, and Dataset: https://github.com/gulraizk94/vlm-smplx-reconstruction