Evaluating the Feasibility of Running AI Large Language Models Locally: Performance, Cost, and Strategic Insights

This article evaluates the use of three different Large Language Models (LLMs) in local deployment scenarios and during experiment phase. Experiments have been selected to include various LLMs and tasks that combine simple textbased tasks, with audio creation and audio transcription services. It measures time of execution, memory consumption and power consumption that results in cost measurements. The article starts by providing background information from the literature, moving into experimental stage and then providing the results. The article discusses the findings and provides feedback in a form of strategic decision that SMEs and or individual researchers can apply.