Image Classification With Unstructured Collections

Most methods for scene understanding in computer vision consider the analysis of a single image or video. However, there is also a long history in multi-view analysis. Having information from multiple views can provide many benefits, such as estimating depth, mitigating issues of occlusion, and generally providing more information about a scene. Previous work in multi-view image classification typically focuses on classifying structured collection data. In this paradigm, the key object, feature, or perspective of each image is predetermined and uniform across all collections. Consequently, classification methods for structured collections are engineered to utilize the known relationships between views and are often designed for specific tasks. In contrast, there has been comparatively less research surrounding the classification of collections where the images are loosely organized and exhibit greater variability, common in tasks such as scene identification, necessitating the development of more flexible approaches. Beyond the image level, unstructured collections can also be applied at the part level, particularly in fine-grained classification tasks. By representing classes as unstructured sets of parts, independent of their source images, classification can be driven by localized feature correspondences across the class rather than relying on global image similarity. This approach mitigates challenges associated with high intra-class variability and enables more robust classification, even in cases where training and test images share only partial visual overlap. In this thesis, we explore the classification of unstructured collections from both perspectives. We begin with introducing a novel approach for classification of unstructured image collections, demonstrating the success of our approach on complex scene identification tasks. We then shift our focus to fine-grained, few-shot classification, introducing a new method to tackle challenging domains characterized with high intra-class variance by matching unstructured part collections. Finally, we unify both approaches into a comprehensive framework, applying part-based matching to refine initial predictions for large-scale tasks, providing a scalable and flexible solution for improving the accuracy of both single-view and multi-view classification systems.