Activation Functions Considered Harmful: Recovering Neural Network Weights through Controlled Channels

Recent advancements in hardware-based enclaved execution environments, such as Intel SGX, aim to protect critical model parameters in high-stakes machine learning applications increasingly moving to end-user or cloud environments. However, this also introduces the risk of privileged side-channel attacks, traditionally aimed mainly at cryptographic targets. In this paper, we develop a novel attack methodology that exploits input-dependent memory access patterns in common neural network activation functions to extract hidden model parameters. In several case studies using the SGX-Step attack framework and the Tensorflow Microlite library, we demonstrate complete recovery of first-layer weights and biases, along with partial recovery of deeper layer parameters under specific conditions. Our novel attack technique requires only 20 queries per input per weight to obtain all first-layer weights and biases, with an average absolute error of less than $1 \%$, improving over prior model stealing attacks. Furthermore, a broader ecosystem analysis reveals the widespread use of activation functions with input-dependent memory access patterns in popular machine learning frameworks and maths libraries. Our findings highlight the limitations of deploying confidential models in SGX enclaves and emphasise the need for stricter side-channel validation of machine learning implementations, akin to the vetting efforts applied to secure cryptographic libraries.