Enhanced Federated RL with NEAT, Markov Chains, and Gaussian Processes

Reinforcement Learning (RL) is a machine learning approach in which an agent learns to make decisions in an environment to maximize a cumulative reward. When combined with NeuroEvolution of Augmenting Topologies (NEAT), RL offers several advantages. NEAT is a genetic algorithm that optimizes the development of artificial neural networks by modifying both their structure and weights. When integrated with RL, NEAT can improve the learning process by merging evolutionary optimization with RL techniques. NEAT has demonstrated significant potential in evolving neural networks for RL tasks. However, traditional centralized training methods encounter scalability and data privacy issues. This paper investigates the integration of NEAT with Federated Learning (FL) and its enhancement with Markov Chains and Gaussian Processes to address certain issues. We propose a new framework that combines NEAT for neural network evolution with TensorFlow Federated (TFF) for decentralized training across multiple clients. Our approach is assessed using the BipedalWalker-v3 environment from OpenAI Gym. The experimental results show that our federated NEAT framework, augmented with Markov Chains and Gaussian Processes, achieves competitive performance while maintaining data privacy and reducing computational overhead on central servers. Additionally, we implement parallelization techniques using concurrent futures to enhance the efficiency of NEAT generations.