A Routing and Cache Management Framework Based on Adaptive Q-Learning for Marine Opportunistic Networks

Marine opportunistic networks are characterized by highly dynamic topology, intermittent connectivity, and severe resource constraints. Traditional routing protocols that rely on fixed-parameter Q-learning cannot adapt to real-time network changes, leading to suboptimal performance. This paper proposes an adaptive framework with three novel contributions: (1) a dynamic learning rate that adapts to network scale, node load, and congestion; (2) a dynamic discount factor that adjusts according to message urgency, hop count, and node mobility; (3) a multi-dimensional reward function with sliding window weights to balance delay, hop count, and node reliability. An asynchronous double Q-learning structure further mitigates overestimation bias. Extensive simulations on the ONE platform demonstrate that the proposed integrated algorithm (IR-DQ) achieves a high delivery ratio, significantly outperforming Epidemic and Spray and Wait, while substantially reducing overhead compared to fixed-parameter Q-learning. The framework exhibits superior adaptability to dynamic marine environments.