Evaluation of LLM Powered Agentic AI for Solving Multi-Arm Bandit Problems

Multi-armed bandit (MAB) problems are foundational to decision-making under uncertainty and have found renewed importance in applications such as online recommendation, adaptive experimentation, and real-time bidding. While classical statistical and reinforcement learning-based algorithms have demonstrated effectiveness, they often require pre-specified exploration strategies and fixed reward priors. With the advent of large language models (LLMs), a new paradigm referred to as agentic AI, has emerged, enabling reasoning-driven problem-solving via natural language interaction. This paper explores the capability of agentic AI to autonomously learn and act in MAB environments using only language-based prompts, without access to reward priors. We evaluate a range of agentic models, including GPT-4o, GPT-4o-mini, and DeepSeek-V3, across diverse bandit configurations that vary in reward distribution complexity. Results indicate that LLM agents can closely approximate optimal behavior in low-entropy settings and demonstrate learning behavior in more complex environments, especially when tuned with appropriate temperature parameters. This work provides an early benchmark for the feasibility and reliability of agentic AI in reinforcement learning settings.