On the generalizability of iterative patch selection for memory-efficient high-resolution image classification

Abstract Classifying large images that contain only small or sparse regions of interest (ROI) remains a fundamental challenge due to computational and memory constraints. Weakly supervised, memory-efficient approaches such as Iterative Patch Selection (IPS) offer an appealing balance between performance and efficiency, yet they struggle to generalize under low signal-to-noise conditions. This study investigates the generalizability limits of IPS using an extended version of the Megapixel MNIST benchmark. We introduce a controlled testbed with tunable object-to-image ratios and a Bézier-curve–based noise generator that better mimics real-world visual clutter. Across tasks, we quantify how generalization varies with object-to-image ratio, training data size, task complexity, and the object-to-patch scale, and we show that increased noise–ROI similarity can precipitate non-convergence. These observations are consistent with a simple attention-collapse account under information sparsity and yield practical guidelines for configuring memory-efficient patch-based classifiers in weakly supervised high-resolution settings.