Compositional and adaptive visual recognition under domain shift

Human vision is remarkably robust, data-efficient, and adaptive. People can recognize objects across changes in style, viewpoint, lighting, modality, and context, often from only a few examples. Modern deep learning systems, despite their impressive performance, still depend heavily on large labeled datasets and often degrade when the test distribution differs from the training distribution. This brittleness is especially consequential in settings where labels are scarce, data are decentralized, or privacy constraints prevent the construction of large centralized datasets. This thesis argues that robust visual recognition requires moving beyond the dominant view of images as global patterns mapped directly to labels. Instead, recognition should be grounded in the structure of visual evidence. Objects and scenes are composed of parts, regions, relations, and context-dependent cues that are not equally relevant to recognition. Inspired by biological perception, the thesis organizes this view around three computational principles. First, visual representations should expose compositional structure rather than collapse images into monolithic embeddings. Second, recognition should selectively compare the evidence that is shared, discriminative, and stable while suppressing clutter, background, and nuisance variation. Third, inference should adapt to the input, domain, or task without requiring full retraining or access to target-domain supervision. The thesis develops architectures and learning strategies that encourage part–whole and region-level structure; training-free and few-shot inference methods that compare images through selected local evidence rather than global similarity alone; adaptive specialization mechanisms that preserve pretrained knowledge while changing the effective computation for new domains; and robustness-oriented optimization strategies that reduce reliance on nuisance-sensitive cues. Although these methods differ in form, they share the goal of making recognition depend less on superficial appearance correlations and more on structured, reusable, and task-relevant visual evidence. The thesis shows that compositional structure, selective evidence use, and adaptive inference provide a coherent foundation for recognition under domain shift, data scarcity, and privacy constraints. Across natural images, medical imaging, federated learning, cross-domain few-shot recognition, and video understanding, the work demonstrates that robust visual recognition is not only a matter of larger models or more data, but of representing and using visual evidence in a more structured and adaptive way.