Adaptive Chain-of-Thought Length Control for Cloud-Based Llms

Chain-of-Thought (CoT) prompting significantly enhances the reasoning capabilities of large language models (LLMs) by encouraging step-by-step problem solving. However, CoT reasoning is often verbose and computationally inefficient, especially when applied uniformly across tasks of varying complexity. Most existing models lack the ability to adapt their reasoning length based on task difficulty, leading to overthinking on simple problems and underperformance on complex ones. We introduce a novel framework that equips LLMs with a CoT Length Controller (CoT-LC)-a lightweight module that predicts a tokenlength budget tailored to each input. This enables a new form of zero-shot + constraint prompting, where the model is guided to generate a reasoning trace of appropriate length without relying on exemplars. Our approach allows the model to “think lighter” on simple questions and “think deeper” on complex ones. Experiments on GSM8K and MATH benchmarks show that our method consistently improves accuracy while reducing or reallocating reasoning tokens effectively. CoT-LC outperforms several strong baselines, demonstrating that dynamic length control is both an efficient and principled strategy for optimizing LLM reasoning.