Enhancing HLS Performance Prediction on FPGAs Through Multimodal Representation Learning

The emergence of design space exploration (DSE) technology has reduced the cost of searching for pragma configurations that lead to optimal performance microarchitecture. However, obtaining synthesis reports for a single design candidate can be time-consuming, sometimes taking several hours or even tens of hours, rendering this process prohibitively expensive. Researchers have proposed many solutions to address this issue. Previous studies have focused on extracting features from a single modality, leading to challenges in comprehensively evaluating the quality of designs. To overcome this limitation, this letter introduces a novel modal-aware representation learning method for the evaluation of high-level synthesis (HLS) design, named MORPH, which integrates information from three data modalities to characterize HLS designs, including code, graph, and code description (caption) modality. Remarkably, our model outperforms the baseline, demonstrating a 6%–25% improvement in root mean squared error loss. Moreover, the transferability of our predictor has also been notably enhanced.