Enhancing on-Device Inference Security Through TEE-Integrated Dual-Network Architecture

Trusted Execution Environment(TEE) offers a secure data processing zone for model inference. Due to its limited resources, existing solutions like MirrorNet usually deploy a lightweight model within the TEE for sensitive data, and a backbone model outside for the rest. However, these approaches do not inherently limit the learning ability of the backbone model, which could acquire inference capabilities similar to the lightweight model, inevitably weakening the security. To counter this, we propose FakeNN, an innovative mechanism with a dual-network architecture, that intentionally guides the backbone model towards low predictive performance, thereby reducing its ability to infer sensitive information. We further improve the accuracy of the entire model by integrating a channel attention mechanism which reduces the transmission of redundant information. We conduct extensive experiments and the results demonstrate that FakeNN substantially expands the performance gap between the non-secure and secure TEE models, with improvements ranging from 3.16% to 66.42% compared to MirrorNet. This enhancement strengthens the system's security without negatively impacting the accuracy of the model.