Crop-and-Prompt: Multi-Grained Prompting for Fine-Grained Visual-Language Understanding

Large Vision-Language Models (LVLMs) have recently achieved remarkable success across a wide range of vision-language tasks, demonstrating impressive capabilities in integrating visual and textual modalities. Despite these advancements, existing open-source LVLMs still exhibit notable limitations when it comes to fine-grained visual perception-a crucial ability for tasks such as distinguishing between visually similar objects, capturing subtle visual cues in high-resolution images, and reasoning about intricate object relationships. To address these challenges, we propose Crop-and-Prompt, a novel inference-time method designed to enhance the fine-grained perception capabilities of LVLMs through multi-grained prompting. The proposed method is composed of two key modules: an image cropping module that systematically extracts high-detail visual regions from the input image, and a multi-grained prompting module that presents the LVLM with a sequence of cropped sub-images alongside a global, compressed view. This design encourages the LVLM to attend to both local details and holistic context, thereby improving its interpretability of complex visual scenes. We evaluate our approach on several benchmark vision-language tasks, demonstrating that Crop-and-Prompt consistently enhances the fine-grained visual perception performance of various open-source LVLMs. Notably, our method is parameter-free, incurs low additional computational cost, and exhibits strong generalizability across models.