Autonomous Decision-Making in Cyber-Physical Systems Using Double Deep Q-Learning and Multi-Agent Collaboration

The autonomous decision-making capabilities of cyber-physical systems (CPS) establish key elements that optimize the management of resources while distributing tasks and maximizing system operational efficiency. Deep Q-Learning (DQN) represents one approach along with Twin Delayed Deep Deterministic Policy Gradient (TD3) and Multi-Agent Proximal Policy Optimization (PPO) that demonstrates capability in handling decision tasks. The existing models face multiple implementation challenges including inaccurate Q-value estimations weak multi-agent relationships and restricted scalability for dynamic systems. The inadequate performance of present methods for energy efficiency and latency alongside poor collaborative coordination led to the necessity of developing a new approach. The new framework DDQN-MAC takes double-deep Q-Learning and combines it with multi-agent collaboration technology to enhance decision-making abilities specific to challenging environments. The method solves overestimation bias through separate action selection and evaluation processes that use multi-agent collaboration for dynamic task distribution and resource control. The designed model demonstrates enhanced performance in multiple critical evaluation measures. Benchmarks indicate that DDQN-MAC attained a superior completion rate of 95.4% while demonstrating reduced latency performance at 12.5 ms versus the other methods DQN (18.7 ms) and Multi-Agent PPO (92.3%). The research provides fundamental groundwork to guide upcoming investigations in hybrid model development while advancing complex multi-agent system capabilities for enhanced autonomous navigational choices within variable operational environments.