Can Small Language Models Replace Traditional Chatbots? A Benchmark Study on 3B LLMs for Local Deployment

This work presents a comprehensive benchmark of three state-of-the-art 3B small language models—LLaMA 3.2, Qwen 2.5, and Phi 3.1 Mini—evaluated entirely in a local, offline environment using GGUF quantized weights and KoboldCPP on consumer hardware (RTX 2060). The study assesses model performance across five real-world task categories: customer support, FAQ answering, short text generation, evaluation of written content, and summarization. Metrics include accuracy, consistency, writing quality, hallucination resistance, and response time. Results show that LLaMA 3.2 delivers the strongest overall performance, with Qwen 2.5 following closely, while Phi 3.1 Mini provides acceptable but more variable results. The findings highlight the growing viability of small language models as efficient, privacy-preserving alternatives to cloud-based chatbots and demonstrate their practical usability for edge AI applications.