Onderzoek rond Taalgedreven Navigatie: van Methoden tot Genuanceerde Evaluatie

With advancements in computer science, the expectations for intelligent agents have evolved beyond merely processing input instructions. We now envision agents capable of perceiving their surroundings through 'eyes,' understanding scenes, objects, and events, and engaging in natural human-like communication. This dissertation explores such agents through the lens of Vision-Language Navigation (VLN), where multimodal representation learning plays a central role. VLN requires agents to interpret visual scenes, process natural language instructions, and perform goal-oriented actions in diverse environments. This work is divided into two parts: methods and evaluation. In the methods section, we investigate VLN under two distinct settings: global information awareness, where agents operate in known environments, and local information reliance, where agents rely on heuristics derived from data in unseen environments. In the evaluation section, we propose a fine-grained framework to diagnose and benchmark VLN models, addressing shortcomings in current evaluation approaches. This research contributes to advancing both the capabilities and assessment of embodied AI systems, offering insights into developing robust and adaptable agents for real-world applications.