Design of 3D Hand Mesh Reconstruction from Monocular Image

Abstract In recent years, as the impact of deep learning on people's lives has grown, more and more attention has been paid to the development of this field. The task of human hand pose and shape estimation from an RGB image has been a long-standing problem in the field of computer vision. Unlike common hand posture prediction, which only predicts the coordinates of the skeletal points of the hand, this task restores the original shape of the hand. Many places will apply this task such as augmented reality and virtual reality, but the task is still very challenging because the hand occupies a relatively small part of the image area, and the hand movements are flexible and easy to block. In this paper, we propose an end-to-end network for 3D hand mesh reconstruction from RGB images. Using ResNet-50 as the encoder, we extract image features and generate 2D heatmaps and masks through convolutional layers to improve parameter regression. The model parameters are iteratively refined with fully connected layers. To correct unnatural hand poses, we apply a graph convolutional network to adjust mesh coordinates, combining these adjustments with the initial MANO-generated model. Experiments demonstrate effective 3D hand mesh reconstruction from monocular images.

Authors

Institutions

Publication Details

Journal
Research Square
Published
2026-11-17
DOI
https://doi.org/10.21203/rs.3.rs-8341709/v1
Primary Topic
Human Pose and Action Recognition
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Design of 3D Hand Mesh Reconstruction from Monocular Image

Tsung‐Han Tsai, Jiayang Jiang
3 links
Research Square
Human Pose and Action Recognition
preprint

Design of 3D Hand Mesh Reconstruction from Monocular Image

Tsung‐Han Tsai, Jiayang Jiang
preprint en

Abstract

Abstract In recent years, as the impact of deep learning on people's lives has grown, more and more attention has been paid to the development of this field. The task of human hand pose and shape estimation from an RGB image has been a long-standing problem in the field of computer vision. Unlike common hand posture prediction, which only predicts the coordinates of the skeletal points of the hand, this task restores the original shape of the hand. Many places will apply this task such as augmented reality and virtual reality, but the task is still very challenging because the hand occupies a relatively small part of the image area, and the hand movements are flexible and easy to block. In this paper, we propose an end-to-end network for 3D hand mesh reconstruction from RGB images. Using ResNet-50 as the encoder, we extract image features and generate 2D heatmaps and masks through convolutional layers to improve parameter regression. The model parameters are iteratively refined with fully connected layers. To correct unnatural hand poses, we apply a graph convolutional network to adjust mesh coordinates, combining these adjustments with the initial MANO-generated model. Experiments demonstrate effective 3D hand mesh reconstruction from monocular images.

Research Square
National Central University (TW)
Human Pose and Action Recognition
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon