Methodologies for Selecting Optimal Hardware for Locally Deployed LLMs Using a Performance, Accuracy and Cost (PAC) Approach

The rapid advancement of Large Language Models (LLMs) has significantly impacted computational linguistics, transitioning from predominantly closed-source, cloud-based models to open-source solutions deployed on edge devices. This shift addresses concerns over data privacy, infrastructure costs, and vendor lock-in but introduces unique challenges for deploying LLMs on resource-constrained devices. This paper presents a methodology for selecting cost-effective hardware platforms for LLM deployment, focusing on a balanced assessment of Performance, Accuracy, and Cost (PAC). Our approach reveals that while traditional methods focusing on performance and cost alone may lead to suboptimal choices, incorporating accuracy into the PAC framework provides a more comprehensive evaluation. By applying this methodology, we offer practical guidance for selecting optimal infrastructure, ensuring that edge deployments are both efficient and effective for various computational linguistics applications.