Digital Twin Assisted Performance Aware Power Management for FPGA MPSoC using High Speed Reinforcement Learning

Performance-aware power manager development poses severe challenge in FPGA MPSoC. Conventional power managers are unaware of the underlying application’s performance requirements. The aspect of application performance awareness can be effectively handled by algorithms based on Reinforcement Learning (RL). The problem though is adaptability to new applications. To adjust to new application scenarios, RL-based algorithms need to be retrained but this causes deadline misses. This motivated us to propose a power manager based on RL assisted by a digital twin framework. The proposed architecture offloads the RL training process to the digital twin stage which emulates reward computation and state estimation. Performance-aware RL training aids in choosing the best frequency and core combination which uses less energy than the current approaches. The proposed methodology is implemented on the FPGA MPSoC ZCU104 with Mi Benchmarks and NATS Benchmarks. It reduces the FPGA energy consumption by 31.395% compared to the FPGA Built-in power manager. Retraining episodes on Deployed FPGA MPSoC are completely eliminated and the convergence time of RL for new workloads decreased to 86 seconds which is significantly less compared to state-of-the-art RL-based methods.