Towards Real-Time and Personalized Code Generation

Large language models (LLMs) have transformed automated code generation. However, their high computational demands often lead to server overload and increased latency in SaaS deployments. To address this, we present SpeCoder, a framework that accelerates server-side code generation using speculative sampling (SpS) and supervised fine-tuning (SFT). SpS allows lower latency in the code generation, whereas SFT enables more personalized code generation tailored to the user's needs.