Q-AIMD: AIMD-based Window Flow Control with Reinforcement Learning

Research on congestion control algorithms, which utilize machine learning techniques without prior knowledge of the network’s internal state, has been actively performed. In recent years, various researchers have proposed Reinforcement Learning (RL) based congestion control algorithms. In the RL-based approach, the window size of the data sender is adjusted according to the reward value obtained from the environment. While recent RL-based algorithms (such as Owl, QTCP, and MVFST-RL) adjust the window size, they either rely solely on incremental changes or the additive increase or multiplicative decrease. The specific values for these adjustments are determined through empirical evaluation. We suggest that modeling the behavior of TCP/IP congestion control algorithms (such as the AIMD algorithm) as action sets and incorporating them into the Q -learning framework can lead to more effective outcomes. Furthermore, these algorithms assume that all network flows are greedy. In real-world networks, flows are not homogeneous and every flow requires different QoS (Quality of Service) requirements. The objective of this research is to propose a Q-learning-based AIMD window flow control (Q-AIMD), a congestion control algorithm that dynamically adjusts the window size using Q-learning in an AIMD manner and also supports heterogeneous flows with different $\mathrm{Q o S}$ requirements.