GRWO: Toward Efficient Model Protection of Edge Inference via Very Few Weights Obfuscation Based on Gradient Ranking

The edge inference of deep neural networks (DNNs) raises considerable concerns regarding the security of DNN models. Using trusted execution environments (TEEs) to isolate model inference and thus protect model privacy has become a leading technology trend. However, existed methods for isolating entire neural network layers are constrained by the memory limitation of TEE. Meanwhile, the obfuscation of partial weights encounters challenges such as the complexity of weight selection and the high recovery overhead due to an excessive number of obfuscated weights. To address these issues, this paper introduces a novel two-phase global weight selection approach based on gradient ranking, designed to achieve optimal model protection with minimal obfuscation. The adversarial attack is also used to guide the weight noise processing, thereby greatly protecting the stealthiness of obfuscated weights. We validated the method on ARM TrustZone and optimized the memory allocation of the model in TEE. Experimental results demonstrate the effectiveness of our method, e.g., by obfuscating 115 out of 3.52 million weights (0.003%), the accuracy of the obfuscated MobileNet-V2 model on ImageNet drops to 0.1%. While the end-to-end latency of the method in this work is comparable to the state-of-the-art solutions, the TEE memory overhead is reduced by 64%.