Efficient Q-learning hyperparameter tuning using FOX optimization algorithm

Reinforcement learning is a branch of artificial intelligence in which agents learn optimal actions through interactions with their environment. Hyperparameter tuning is crucial for optimizing reinforcement learning algorithms and involves the selection of parameters that can significantly impact learning performance and reward. Conventional Q-learning relies on fixed hyperparameter without tuning throughout the learning process, which is sensitive to the outcomes and can hinder optimal performance. In this paper, a new adaptive hyperparameter tuning method, called Q-learning-FOX (Q-FOX), is proposed. This method utilizes the FOX Optimizer—an optimization algorithm inspired by the hunting behaviour of red foxes—to adaptively optimize the learning rate ( α ) and discount factor ( γ ) in the Q-learning. Furthermore, a novel objective function is proposed that maximizes the average Q -values. The FOX utilizes this function to select the optimal solutions with maximum fitness, thereby enhancing the optimization process. The effectiveness of the proposed method is demonstrated through evaluations conducted on two OpenAI Gym control tasks: Cart Pole and Frozen Lake. The proposed method achieved superior cumulative reward compared to established optimization algorithms, as well as fixed and random hyperparameter tuning methods. The fixed and random methods represent the conventional Q-learning. However, the proposed Q-FOX method consistently achieved an average cumulative reward of 500 (the maximum possible) for the Cart Pole task and 0.7389 for the Frozen Lake task across 30 independent runs, demonstrating a 23.37% higher average cumulative reward than conventional Q-learning, which uses established optimization algorithms in both control tasks. Ultimately, the study demonstrates that Q-FOX is superior to tuning hyperparameters adaptively in Q-learning, outperforming established methods. • Introducing Q-learning-FOX, an adaptive method for tuning hyperparameter in Q learning, known as Q-FOX. • Proposes a novel objective function combining average Q-values and learning error for efficient learning process. • Demonstrates that the Q-FOX outperforms Q-learning, which uses established optimization algorithms to maximizing reward.