A Benchmark Dataset for Evaluating Spatial Perception in Multimodal Large Models

Multimodal large language models (LLMs) are increasingly viewed as potential central processors for embodied intelligence, where spatial perception---such as arranging objects and understanding distance, size, and volume---is crucial. Existing datasets often suffer from data leakage and may not effectively benchmark new multimodal models. In this work, we introduce a scalable dataset synthesized on demand, eliminating repeated data and allowing controlled difficulty levels. The dataset encompasses 9 easy to scale categories. Evaluation with a commercial LLM and a open source LLM reveals that current LLMs exhibit limited proficiency in reasoning across two-dimensional spaces, underscoring the need for improvement in spatial reasoning capabilities.