Adaptive Chain-of-Thought Distillation Based on LLM Performance on Original Problems

The chain-of-thought (CoT) approach in large language models (LLMs) has markedly enhanced their performance on complex tasks; however, effectively distilling this capability into LLMs with smaller parameter scales remains a challenge. Studies have found that small LLMs do not always benefit from CoT distillation. Inspired by the concept of teaching students in accordance with their aptitude, we propose an adaptive chain-of-thought distillation (ACoTD) framework. The core idea is to dynamically and adaptively customize distillation data and supervision signals for student models based on their performance on the original problems. Specifically, ACoTD initially evaluates and categorizes the original problems according to the capabilities of the student model. Subsequently, for Easy- and Medium-level problems, a short CoT distillation is employed for a brief lecture to reinforce knowledge and enhance training efficiency, for high-difficulty problems where the student model underperforms, and a detailed long CoT distillation is utilized for in-depth explanation to infuse richer reasoning logic. This differentiated distillation strategy ensures that student models achieve a better grasp of learning. We conducted experiments on multiple benchmark datasets. The results indicate that, compared to the baseline, our method can significantly improve the inference performance of small LLMs. Our method provides a new student-centered paradigm for knowledge distillation, demonstrating that adaptive adjustment of teaching strategies based on student feedback is an effective way to enhance small LLMs’ reasoning ability.