Extern Kennis Benutten in Diep Leren en Conditionele Generatieve Modellen

This PhD project explores advancing AI systems toward human-like intelligence through integrating external knowledge and refining generative models. The first part focuses on embedding various external knowledge types, such as linguistic data for enhanced language understanding and structured triplet knowledge for improved visual question answering. It also investigates leveraging commonsense visual knowledge from pretrained text-to-image models to support tasks like remote object grounding, demonstrating how external information can boost model comprehension and performance. The second part concentrates on refining generative diffusion models to produce high-quality visual content tailored to human needs. This includes incorporating guiding signals, such as text, images, and noisy data like fMRI signals, to enhance output control. A proposed time-shifted algorithm addresses inherent diffusion model limitations, mitigating exposure bias to improve content generation. Overall, these approaches show that integrating external knowledge and innovative signal guidance can significantly enhance AI capabilities, moving them closer to human-level intelligence.