An Improved Q-Learning-Based AODV Routing Protocol for Marine Cross-Medium Acoustic–Radio Collaborative Networks

Marine cross-medium acoustic–radio collaborative networks must route traffic across heterogeneous underwater acoustic and radio links while coping with energy imbalance, congestion, and mobility-induced link instability. This paper proposes Q-Learning AODV, an AODV extension that integrates distributed Q-value updating and multipath route maintenance into existing RREQ, RREP, and HELLO procedures. The routing reward combines normalized residual energy, queue availability, inter-node distance/link stability, relative velocity, and, for air–sea links, elevation-angle information. The protocol maintains multiple node-disjoint candidate paths and forwards data through the currently highest-valued path. NS-3 simulations are reported for underwater-to-underwater, underwater-to-air, and air-to-underwater communication scenarios. Relative to conventional AODV, Q-Learning AODV increases packet delivery ratio from 55.8% to 88.3%, from 68.3% to 76.1%, and from 86.9% to 91.9%, corresponding to relative improvements of 58.2%, 11.4%, and 5.8%, respectively. The results indicate improved delivery reliability and communication-subsystem energy balancing at the cost of additional state exchange, Q-table storage, and route-selection computation.