The Fragility of Optimal-Agent Training: Continuous-Time Dynamics and Adaptive Computation as Heuristic Regularizers in Edge AI

We present a series of counterintuitive empirical findings that challenge fundamental assumptions in reinforcement learning (RL) for Edge AI. Training lightweight recurrent agents—based on Closed-form Continuous-time Neural Networks (CfC) and Gated Recurrent Units (GRU)—against a near-perfect Minimax opponent in Tic-Tac-Toe produces policies that achieve a 100% draw rate against the optimal adversary, yet collapse catastrophically when confronted with stochastic opponents. Our ablation study reveals three pathologies: (i) a hardware paradox wherein Adaptive Computation Time (ACT) increases wall-clock training time by 2.45x despite reducing theoretical ODE integration steps; (ii) latent-space fragmentation producing non-transitive dominance cycles—a GRU agent winning only 11% against random defeats a CfC+ACT agent 100-0; and (iii) negative transfer from Tic-Tac-Toe to Connect Four, degrading performance from 80% to 23% vs. from-scratch training. We argue that continuous-time dynamics and adaptive halting function primarily as heuristic regularizers whose benefits are contingent on deployment constraints, and that the pursuit of optimality in self-play constitutes a structural trap for Edge AI systems.