Hybrid Q-Learning with VLMs Reasoning Features

This paper introduces a novel hybrid reinforcement learning framework that integrates Vision Language Models perception, symbolic reasoning, and Q-learning for navigation in grid world environments. Our architecture combines a VLMs-based Reasoning module for extracting visual features, a symbolic extraction component for structured representation, a neural reasoning module for logical inference, and a Q-network for action selection. We evaluate the framework on a 5x5 grid world navigation task, analyzing both its learning dynamics and final performance. Results demonstrate significant learning progression, with evaluation rewards improving by 92.6% during training. However, our experiments also reveal important limitations, particularly in the symbolic extraction component, which consistently failed to generate meaningful representations from visual inputs. Final evaluation across 100 test episodes yielded a 37% goal-reaching accuracy, with successful episodes averaging just 2 steps to goal completion. Though the architecture demon-strated capability for efficient navigation in specific scenarios, the overall performance suggests further refinement is needed in the vision-to-symbol mapping process. Our findings highlight both the potential of hybrid approaches of VLMs and the challenges in effectively bridging visual perception with symbolic reasoning in reinforcement learning tasks.