Soft Fuzzy Reinforcement Neural Network Proportional–Derivative Controller

Controlling systems with highly nonlinear or uncertain dynamics present significant challenges, particularly when using conventional Proportional–Integral–Derivative (PID) controllers, as they can be difficult to tune. While PID controllers can be adapted for such systems using advanced tuning methods, they often struggle with lag and instability due to their integral action. In contrast, fuzzy Proportional–Derivative (PD) controllers offer a more responsive alternative by eliminating reliance on error accumulation and enabling rule-based adaptability. However, their industrial adoption remains limited due to challenges in manual rule design. To overcome this limitation, Fuzzy Neural Networks (FNNs) integrate neural networks with fuzzy logic, enabling self-learning and reducing reliance on manually crafted rules. However, most fuzzy neural network PD (FNNPD) controllers rely on mean square error (MSE)-based training, which can be inefficient and unstable in complex, dynamic systems. To address these challenges, this paper presents a Soft Fuzzy Reinforcement Neural Network PD (SFPD) controller, integrating the Soft Actor–Critic (SAC) framework into FNNPD control to improve training speed and stability. While the actor–critic framework is widely used in reinforcement learning, its application to FNNPD controllers has been unexplored. The proposed controller leverages reinforcement learning to autonomously adjust parameters, eliminating the need for manual tuning. Additionally, entropy-regularized stochastic exploration enhances learning efficiency. It can operate with or without expert knowledge, leveraging neural network-driven adaptation. While expert input is not required, its inclusion accelerates convergence and improves initial performance. Experimental results show that the proposed SFPD controller achieves fast learning, superior control performance, and strong robustness to noise, making it effective for complex control tasks.