CoReD: Council-Weighted Reasoning Distillation

Standard chain-of-thought distillation transfers reasoning ability to small models, but typically treats all reasoning steps uniformly despite substantial differences in their importance within a reasoning trajectory. Existing step-aware methods largely use importance signals for hard selection or pruning, leaving the retained steps weighted uniformly. We propose Council-Weighted Reasoning Distillation (CoReD), which derives continuous step weights from a deliberately diversified council of LoRA experts. For each step, we combine the experts' average predictive uncertainty with their normalized disagreement, capturing whether a step is difficult for individual experts or controversial across the council. To make disagreement informative, the experts are explicitly diversified at uncertain reasoning steps before being consolidated in weight space. The merged adapter initializes the student and acts as a frozen distributional anchor during subsequent step-weighted optimization. After training, the learned adapter is merged into the backbone, introducing no additional inference overhead. Across mathematical reasoning benchmarks, CoReD consistently outperforms standard CoT distillation and recent step-aware reasoning methods.