Real Time Sentiment Analysis of YouTube Live Chat With Apache Spark and Kafka

The rapid growth of live streaming platforms has created a demand for systems that can analyze user sentiment in real time during high-impact events. This study presents a scalable architecture for real-time sentiment analysis of YouTube live chat, designed and implemented using Apache Kafka for message streaming, Apache Spark Structured Streaming for processing, and Prometheus–Grafana for system monitoring and visualization. Although Chandrayaan-3 was chosen as a representative test case, the framework is event-agnostic and can be applied to other mass participation contexts such as general elections, sports tournaments, and large-scale product launches, where real-time sentiment insights are equally crucial. A comparative analysis of traditional machine learning models and a deep learning model was conducted, with Logistic Regression achieving the best trade-off between accuracy (99.2%) and latency. While the LSTM model performed less effectively due to dataset size constraints, its inclusion provided valuable insights into the feasibility of deep learning in real-time pipelines. Performance metrics such as accuracy, precision, recall, and latency were validated using cross-validation to ensure robustness. The study also discusses practical limitations, including potential overfitting, the challenge of sarcasm and code-mixed language in sentiment analysis, and the generalizability of results beyond a single live-streaming event. Despite these constraints, the system demonstrates the feasibility of integrating scalable big data frameworks with visualization tools to provide actionable insights in mission-critical live events. Future research will focus on multilingual extensions, larger cross-event datasets, and advanced deep learning models such as transformers to further enhance reliability and adaptability.