Extending Q-Learning Agents in SQLi Environments

Q-Learning is one of the foundational techniques in reinforcement learning and provides a benchmark of comparison for other techniques. Reinforcement learning has been used in a plethora of fields and is being increasingly used in cybersecurity. It is employed to automate penetration testing and perform reconnaissance on vulnerabilities, with SQL injection being a prototypical example. Previous research efforts have focused on creating agents to exploit these vulnerabilities, but how agents should interact with these environments is still being investigated. In this research endeavor, extensions to previous agents using the standard methodology of reinforcement learning will be extended to measure their effectiveness in a complex environment. The following alternative methods are reviewed: hyperparameter tuning, double Q-Learning, N-step Q-Learning, and using Upper Confidence Bound exploration. This evaluation will provide guidance regarding developing future agents. It is of great importance to understand how these refinements to Q-Learning may empower the agent without having to resort to more sophisticated methods. Other methods may require additional trade-offs and may introduce unfavorable dynamics to the environment. By understanding Q-Learning and related methods, it may be possible to avoid using black-box algorithms that would be detrimental to interpretability while still increasing performance.