Capture the Key in Reasoning to Enhance CoT Distillation Generalization

As Large Language Models (LLMs) scale up and gain powerful Chain-of-Thoughts (CoTs) reasoning abilities, practical resource constraints drive efforts to distill these capabilities into more compact Smaller Language Models (SLMs).We find that CoTs consist mainly of simple reasoning forms, with a small proportion ( 4.7%) of key reasoning steps that truly impact conclusions.However, previous distillation methods typically involve supervised fine-tuning student SLMs only on correct CoTs data produced by teacher LLMs, resulting in students struggling to learn the key, instead imitating the teacher's reasoning forms and making errors or omissions in reasoning.To address these issues, drawing an analogy to human learning, where analyzing mistakes according to correct solutions often reveals the crucial steps leading to successes or failures, we propose mistakE-Driven key reasonIng step distillaTion (EDIT), a novel method that further aids SLMs learning key reasoning steps rather than mere simple fine-tuning.Firstly, to expose the crucial steps in CoTs, we carefully design specific prompts to generate dual CoTs data with similar reasoning paths but divergent conclusions.Then, we apply the minimum edit distance algorithm on the dual CoTs data to locate these key steps and optimize the likelihood on these tokens.Extensive experiments and analysis validate the effectiveness of EDIT across both in-domain(IND) and out-ofdomain(OOD) benchmark reasoning datasets 1 .* Kun Li is the corresponding author. 1 Code can be found at https://github.com/ C-W-D/EDIT On the floor, I see two magenta scrunchiephone chargers, one gold fidget spinner, three gold booklets, three silver scrunchiephone chargers, two silver booklets, and two magenta booklets.If I remove all the magenta items from the floor, how many scrunchiephone chargers remain on it?Op ons: (A) zero (B) one (C) two (D) three (E) four (F) five (G) six According to this ques on, there are two magenta scrunchiephone chargers and three silver scrunchiephone chargers on the floor.If we remove all the magenta items from the floor, we are le with 2 silver scrunchiephone chargers.Therefore, the answer is (C).According to this ques on, there are two magenta scrunchiephone chargers and three silver scrunchiephone chargers on the floor.If we remove all the magenta items, we are le with only the silver scrunchiephone chargers.Therefore, the number of scrunchiephone chargers that remain on the floor is three.Therefore, the answer is (D).Teacher Ques on Unthinking Student On the floor, I see two magenta scrunchiephone chargers, one gold fidget spinner...According to this ques on, there are two magenta scrunchiephone chargers and three silver scrunchiephone chargers on the floor.If we remove all the magenta items from the floor, we are le with 2 silver scrunchiephone chargers.Therefore, the answer is (C).According to this ques on, there are a total of 5 scrunchiephone chargers on the floor: 2 magentascrunchiephone chargers and 3 silver scrunchiephone chargers.If we remove all the magenta items from the floor, we remove the 2 magenta scrunchiephone chargers.Therefore, the number of scrunchiephone chargers remaining on the floor is 3. Therefore, the answer is (D).