Training Dynamics of Recurrent Neural Networks in Random-Target Reaching Task

We investigated the training dynamics of Gated Recurrent Unit networks controlling a 2-joint musculoskeletal arm model in a random-target reaching task. Our networks exhibited unusual non-monotonic loss behavior—initially decreasing, then increasing until epoch 100, before finally stabilizing—contrary to the typical monotonic convergence pattern seen in most neural networks. Through systematic parameter optimization, we established an optimal configuration (256 latent size, 1e-3 learning rate with warmup, 1e-6 weight decay) that improved overall performance while the non-monotonic pattern persisted. Using Multidimensional Scaling (MDS) visualizations and Dynamic Similarity Analysis (DSA), we revealed that the network explores distinct computational regimes during training rather than refining a single strategy, suggesting a phase transition-like progression through qualitatively different computational states.