Novel Q-learning-based neighborhood discovery algorithm for directed antennas

Directed antennas offer significant energy savings, yet the neighbor-discovery phase requires nontrivial algorithms. Using graph-theoretic analysis, we establish theoretical lower and upper bounds on the time required to reach minimally connected antenna topologies. We develop a Q-learning-based (QL) neighbor-discovery algorithm and introduce two enhancements. First, we propose an adaptive mechanism that dynamically updates the exploration rate <br/>ɛ based on changes in Q-tables, enabling a principled balance between exploration and exploitation. Second, we introduce a temporal sector-pruning heuristic that penalizes and prunes unfruitful sector pairs. Experiments are performed in a custom simulator, measuring the time required to reach minimal connectivity. We compare the proposed methods against random walk and baseline QL across regular, dense (King Grid), and irregular, sparse (Random) topologies. Results show that the synergistic integration of pruning and adaptive QL reduces discovery time by about one-third relative to QL on regular and dense graphs. On sparse graphs, baseline QL fails to complete within the threshold time, while the synergistic scheme achieves at least a tenfold speed-up. The synergistic method further outperforms pruning-only QL by 30%–86% and adaptive-only QL by 7%–16% in discovery time.