A federated discrete reinforcement learning architecture for enhancing convergence and interpretability in distributed control

Federated reinforcement learning is a promising approach for distributed artificial intelligence in cyber–physical energy systems, but many existing methods rely on black-box deep policies that reduce interpretability and may converge unstably under non-identical operating conditions. This work proposes a federated discrete reinforcement learning architecture for collective pitch control of wind turbines. The method combines federated learning, tabular reinforcement learning, and structured state–action representations so that turbines collaboratively learn transparent state–action value tables instead of opaque neural policies. The architecture is evaluated with multiple wind turbines exposed to independent wind realizations generated from the same Weibull distribution. Several federated aggregation strategies are compared in terms of convergence behaviour, training stability, scalability, synchronization frequency, parameter sensitivity, and communication requirements. The simulations show that the proposed proximal discrete aggregation method reached stable operation after approximately 30 training episodes, whereas Federated Averaging and the non-federated controller required approximately 70 episodes. FedSGD required the longest training time, converging after approximately 450 episodes, and presented the largest fluctuations, as reflected by the 95% confidence intervals. Increasing the number of participating turbines improved the convergence trend. These results demonstrate that discrete federated reinforcement learning can provide an effective balance between interpretability, convergence performance, and distributed learning efficiency in energy control applications.