Enhancing Submarine Torpedo Countermeasure Tactics via Curriculum Reinforcement Learning with a Quantitative Tactical Difficulty Metric

The development of effective torpedo countermeasure tactics is critical for submarine survivability, yet training autonomous agents for this task remains challenging. In this paper, we propose a novel framework that enhances reinforcement learning agents via curriculum learning based on quantified tactical difficulty. First, we quantify tactical difficulty from simulation data, then use grid search to determine an optimal, multi-stage learning schedule. Our model, trained with this curriculum, achieved a final countermeasure effectiveness of 97.94 % in the final performance evaluation, a 3.5 percentage point improvement over a baseline agent trained on a randomized order (94.44 %). Notably, this superior final performance was achieved even when the baseline agent exhibited higher rewards during the training phase. This result indicates that the proposed curriculum is a more effective method for achieving higher final performance than standard randomized training.