MeRM: Large Mesh Reconstruction Model via Neural Geometry Tokens

High-fidelity surface reconstruction from sparse RGB images remains a significant challenge, where classical optimization-based approaches typically fail to resolve geometric ambiguity under insufficient epipolar constraints. While recent feed-forward methods have achieved remarkable progress in directly regressing point clouds and 3D Gaussians from sparse inputs, extending such paradigms to explicit surface reconstruction often results in suboptimal geometric quality. We identify the primary obstacle in feed-forward surface reconstruction as an inherent structural misalignment between 2D visual inputs and 3D surface targets, which creates a learning bottleneck and leads to degraded performance. In this work, we propose MeRM, a large-scale feed-forward Mesh Reconstruction Model that leverages a novel Neural Geometry Token (NGT) technique to address this challenge. Our key insight is that intermediate tokens of a multi-view Transformer—pretrained to predict dense depth maps, normal maps, and masks—can serve as a powerful geometric pre-aggregator to bridge the structural bottleneck. Benefiting from the highly informative geometric cues distilled by NGT, our MeRM delivers exceptional surface reconstruction quality while maintaining impressive training and inference efficiency. Extensive experiments demonstrate that our MeRM significantly outperforms prior feed-forward methods and even exceeds the performance of state-of-the-art optimization-based approaches on dense views, despite using only sparse inputs. We believe our approach establishes a novel framework for large-scale feed-forward mesh reconstruction, paving a new path for future research.