Heterogeneous Confidential Computing System for Large Language Models: A Survey

The rapid progress of large language models (LLMs) has driven their deployment on heterogeneous computing platforms using accelerators such as GPUs and NPUs to meet growing computational demands. However, this paradigm transition introduces critical security and privacy risks that are uniquely intensified by the characteristics of LLM workloads. These models embed high intellectual property value, making them prime targets for adversaries. Furthermore, their autoregressive generation process makes them exceptionally sensitive to latency, magnifying the performance impact of any security overhead. Existing confidential computing solutions are predominantly CPU-centric, leaving critical computation on accelerators unprotected. This article provides a survey of technologies for securing LLMs on heterogeneous confidential computing platforms. We analyze the threat landscape of heterogeneous LLM deployment, identifying risks across hardware interconnects, complex software stacks, and LLM-specific properties. These risks include data leakage during transfer, interference in multi-tenant execution, and limited trust in accelerator components. To address these challenges, we review key enabling technologies including secure I/O, tenant isolation, and device attestation, which together form the foundation for building trusted execution paths across heterogeneous platforms. We evaluate representative academic and industrial systems for GPUs and NPUs, summarizing their protection strategies and current limitations. Finally, we highlight open research directions such as fine-grained protection of LLM runtime state, low-overhead cryptographic engines, and formal methods for building verifiable systems. Our goal is to support secure and efficient LLM execution through the integration of confidential computing as a foundational element in heterogeneous system design.