A First Look At Efficient And Secure On-Device LLM Inference Against KV Leakage
Running LLMs on end devices has garnered significant attention recently due to their advantages in privacy preservation. With the advent of lightweight LLM models and specially designed GPUs, on-device LLM inference has achieved the necessary accuracy and performance metrics.
Authors
- D. H. Zhang (ORCID: https://orcid.org/0000-0003-0298-7868)
- Huan Yang (ORCID: https://orcid.org/0000-0002-6262-9096)
- Yuanchun Li (ORCID: https://orcid.org/0000-0002-1591-2526)
- Yudong Zhao (ORCID: https://orcid.org/0009-0007-5038-7783)
- Yunxin Liu (ORCID: https://orcid.org/0000-0001-7352-8955)
Institutions
- Central South University (CN)
- Tsinghua University (CN)
Publication Details
- Published
- 2024-11-18
- DOI
- https://doi.org/10.1145/3691555.3696827
- Citations
- 3
- Primary Topic
- Cryptography and Data Security
- Type
- article
- Field-Weighted Citation Impact
- 1.04
Funders
- National Science Foundation
- National Natural Science Foundation of China
- Central South University
- National Outstanding Youth Science Fund Project of National Natural Science Foundation of China
- National Key Research and Development Program of China