Edge-Based Platform for Large Language Models Deployment in Internet of Things Applications

Decision-making support systems are a growing trend in Internet of Things (IoT)-based applications. It is due to the introduction of Large Language Model (LLM) deployment platforms. However, cloud-centric approaches struggle with their implementation due to scalability issues. This work proposes an edge-based alternative to improve the scalability of LLM deployment in IoT-based applications. It also presents the incorporation of Raspberry Pis with low-cost, compact, and trending Artificial Intelligence (AI)-based applications, making it an ideal choice as an IoT device for deploying the LLM locally. Experiments evaluate the responsiveness of the LLM under the proposed edge-based platform. The results show its potential in minimizing the likelihood of network scalability issues, while the edge-based approach manages the LLM locally through the Raspberry Pi.