De dynamiek van het leren van visuele representaties zonder labels begrijpen en overdragen

Self-supervised learning (SSL) has revolutionized computer vision by enabling models to learn powerful visual representations without labeled data. However, fundamental questions remain about how these models learn and how their training dynamics—not just their representations—can be effectively transferred across tasks. This project explores the underlying principles that govern SSL training and develops novel methods to adapt these insights for a wide range of applications, from domain adaptation to multimodal learning and active visual exploration. Key contributions of this research include: • Analyzing Augmentation Strategies: We dissect the role of spatial augmentations, such as cropping and cutout, in shaping learned features and introduce a novel distance-based margin for invariance loss, improving representation quality in diverse data distributions. • Unifying SSL Stability Mechanisms: By identifying a common stability principle across contrastive and non-contrastive SSL methods, we provide theoretical insights that inform more robust and efficient training strategies. • Efficient Domain Adaptation (UpStep): We develop a source-free adaptation technique that leverages self-supervision and targeted fine-tuning to reduce training time by 50% while maintaining strong performance across new domains. • Advancing Visual Question Answering (VQA): By integrating redundancy reduction techniques, we enhance the alignment between image-question and answer embeddings, improving both accuracy and interpretability. • Optimizing Active Visual Exploration: We propose novel self-attention-based methods for selecting the most informative glimpses in partially observable scenes, significantly improving scene reconstruction with fewer computational resources. This project demonstrates how a deeper understanding of SSL's training dynamics can drive advancements across multiple areas of computer vision. By applying SSL principles in a structured way, we open up new possibilities for more efficient, adaptable, and scalable learning systems. This approach helps reduce dependence on labeled data and broadens the potential uses of AI in real-world applications.