Space-Filling Input Design for Nonlinear System Identification: A Reinforcement Learning Approach

Classical input design minimises the covariance of the estimated model parameters. For nonlinear black-box models, however, the dominant error is the bias induced by the model structure over the region of interest, which motivates space-filling strategies seeking to cover the system state space as uniformly as possible. Existing approaches rely on predefined input classes and require solving explicit optimisation problems. In this paper, we propose a reinforcement learning (RL) approach to space-filling input design, in which the agent learns a control policy that actively drives the system to explore the state space. Unlike optimisation-based methods, the proposed approach is not restricted to a specific input parametrisation, operates directly under an amplitude bound, and adapts to varying initial conditions through its observation-based policy without re-optimisation. The learned excitation signals exhibit physically meaningful behaviours, such as resonance excitation and amplitude modulation. On two nonlinear benchmark systems, the approach is evaluated against tuned swept sine and crest-factor-optimised multisine excitations, and against a recent space-filling multisine design. Among the inputs respecting the amplitude bound, the learned policy achieves the best space-filling performance; the space-filling multisine reaches comparable or larger coverage only with peaks well above that bound.