A Glimpse to Scalable LLM Architectures: Building Real-Time Sentiment Analyser with Kafka and FastAPI

In today's digital world, the real-time generation and analysis of data, particularly on social media platforms, are becoming increasingly important. This study aims to design and evaluate a scalable architecture for a scenario where multiple users simultaneously perform sentiment analysis. Two different architectures were examined in the study: the Basic Architecture and the Proposed Architecture. The Basic Architecture processes and responds to user requests directly using FastAPI and Gunicorn. In contrast, the Proposed Architecture employs FastAPI, Kafka, and Redis to create a more complex and distributed structure. Test results demonstrate that the Proposed Architecture offers higher throughput and lower response times under high demand and user loads. This study presents significant findings on how to establish a scalable and high-performance infrastructure for large language models.