GEMMify: A Unified Model for Layer-Wise DNN Performance Prediction on Edge GPUs

Performance prediction techniques enable early design space exploration to create and deploy cost-effective Deep Neural Networks (DNNs). Specifically, layer-wise DNN performance predictions produce a fine-grained performance breakdown of DNNs, proving to be valuable for automated model optimization. However, current methods embed a predictive model for each type of DNN layer, which increases their complexity and lowers their practicality when targeting novel neural networks. Using similarities between General Matrix Multiplication (GEMM) operations and main DNN layers, such as convolutional (CONV) and fully connected (FC) layers, we introduce a novel method for predicting the layer-wise power, latency, and energy consumption of DNNs. The proposed method uses GEMM dimensions as common features for the main layers, allowing performance prediction using a single model. Experiments on the Jetson Orin edge device show that the proposed method achieves state-of-the-art Mean Absolute Percentage Error (MAPE) ranging from 9.77% to 15.28% while offering widely applicable models.