Comparing a computational model of visual problem solving with human vision on a difficult vision task

Human vision is not merely a passive process of interpreting sensory input but can also function as a problem-solving process incorporating generative mechanisms to interpret ambiguous or noisy data. This synergy between the generative and discriminative components, often described as analysis-by-synthesis, enables robust perception and rapid adaptation to out-of-distribution inputs. In this work, we investigate a computational implementation of the analysis-by-synthesis paradigm using genetic search in a generative model, applied to a visual problem-solving task inspired by star constellations. The search is guided by low-level cues based on the structural fitness of candidate solutions compared to the test images. This dataset serves as a testbed for exploring how inferred signals can guide the synthesis of suitable solutions in ambiguous conditions, framing visual inference as an instance of complex problem solving. Drawing on insights from human experiments, we develop a generative search algorithm and compare its performance to humans, examining factors such as accuracy, reaction time, and overlap in drawings. Our results shed light on possible mechanisms of human visual problem solving and highlight the potential of generative search models to emulate aspects of this process.