Time-Modulated Hebbian Learning for Classic Control Tasks

Artificial neural networks (ANNs) commonly rely on fixed-weight architectures, limiting their adaptability in dynamic and uncertain environments. In contrast, biological neural systems exhibit plasticity, enabling continuous adaptation to changing conditions. Hebbian learning provides a mechanism for incorporating plasticity into ANNs, offering the potential for improved flexibility in real-world control tasks. However, Hebbian-based models often suffer from instability, making their practical application challenging. This work explores the trade-off between adaptability and stability in Hebbian learning-driven ANNs by introducing a neuro-modulatory term into the standard Hebbian learning formulation, enabling a gradual transition from high plasticity to fixed weights over time. The proposed model, named time-modulated Hebbian learning (tm-HL), was tested on three classical control tasks from the Gymnasium environment, and it was compared to a standard Hebbian learning model and a fixed-weight ANN. An evolutionary algorithm was employed to optimise the Hebbian parameters for the former and the weights of the latter. The proposed approach showed a promising performance, showing higher adaptability over the model with fixed weights in certain scenarios. Moreover, it consistently performed better than the standard Hebbian learning model in all the testing environments.