An Automatic Framework for Collaborative CPU Thread Throttling and FPGA HLS-Versioning

Cloud environments have widely adopted collaborative CPU-FPGA architectures to speed up applications by executing kernels across both devices. However, optimizing the execution on these architectures is challenging and requires smart utilization of optimization techniques depending on the application and target optimization, such as performance or energy. Additionally, incoming application requests to the Cloud with different priorities result in workloads in the form of directed acyclic graphs, which imposes an additional layer of complexity on the optimization process. To address these challenges, we propose a framework that employs Thread Throttling (i.e., artificially decreasing the number of active threads) on the CPU side and HLS (High-Level Synthesis)-versioning on the FPGA side to improve the performance and energy efficiency of CPU-FPGA Clouds. Our framework uses an offline Design Space Exploration to collect execution time and energy consumption data of kernels across a wide range of CPU and FPGA configurations. This data is used at runtime to properly apply Thread Throttling and HLS-versioning on the kernels to optimize overall execution. Our findings show that by synergistically applying Thread Throttling and HLS-versioning to the incoming kernels, the framework can improve the energy-dealy product by up to 19.5x over the default and non-optimized execution.